Guest: Robert Holcomb - SOASTA

Speaker
So, he's Robert Holcomb.

Speaker
Hey, how are you doing?

Robert Holcomb - SOASTA
(Inaudible) on that.

Speaker
Mikes are always a whole lot of fun in the video business. So, what do you do?

Robert Holcomb - SOASTA
I'm the Director of Product Management, among many other things.

Speaker
So, you probably get a pizza, right.

Robert Holcomb - SOASTA
Not yet, but I am sure that's probably in my future.

Speaker
Yeah, he actually installs solves our network, edits our Video Podcast, he is one of the performers in the Video Podcast, and he is our Sales Engineering Consultant that's doing all of our -- helping our alpha customers bring the product up. So, one of the really good things about startups is that you get to do lots of stuff.

Speaker
Oh, yeah.

Speaker
So, why don't you really quickly show him this.

Robert Holcomb - SOASTA
Okay. So, again it's been mentioned a number of times, but really the emphasis is on making a graphical...

Speaker
So, what are we looking at?

Robert Holcomb - SOASTA
So, what we are looking at right here is a sample task, a sample unit test that I've created to test the Salesforce.com Web Service.

Speaker
So, this is the inside of one of those message clips, okay, the ones that we put on that multitrack interface. This is how you create the inside of that.

Robert Holcomb - SOASTA
Exactly, so this particular clip is doing a specific test to login to the salesforce.com web service, creates some accounts -- clear to ensure that those accounts were created properly and then just delete them out. So, a pretty straightforward test, really just to test a lot of different functionality of the Salesforce.com service itself.

So, walking through how we might be able to create one of these things pretty straightforward. You would first go and create a target, what we call a 'Target,' which is essentially your connection to the Destination Web Service. You can see here, I have a number of them already created against Amazon and Google and Salesforce. But going in and creating a target is simple in our environment as either pasting in the URL to the WSDL location or pasting in the WSDL itself into this part of the Target Editor. So, with getting to WSDL in here, now we know everything we need to know about this particular web service. We know all the different operations it supports and essentially the test that we need to create to test this particular web service. So, I will attach an image to this particular target for reasons you actually saw on the previous screen. And I can go ahead and just give this -- save this. Okay, well we look at one that's already been saved. How about that? So anyone of these that we've already created so the Salesforce.com, one that has the enterprise. Same exact thing has all the WSDL in it with the logo...

Speaker
Now, as I was mentioning earlier Salesforce.com has 63 data objects and 31 message operations that you can form against each one of those data objects. So, this is certainly not the biggest web service interface you'll see, it's pretty compact and it's incredibly well-designed. We're big admirers of what they've done. But you end up with a lot of permutations very, very quickly and so, the average person the average Java developer or .NET developer isn't walking around with an understanding of what all of those verb object combinations do and so what you want to be able to do in our environment is to bring in the business analyst who does understand, each one of those messages is essentially a transaction. And so you want somebody who understands, how to string those transactional actions together in order to do something useful in the system, so that's the idea here.

Robert Holcomb - SOASTA
Okay. So, once you get the target created, now you can create any number of different unit tests, which we call Message Clips in our environment. So, again that's a point -- we have a point-and-click interface using graphical forms using the AJAX technology. So, as Ken showed earlier, creating a new message which typically when I was doing testing with Pro and XML Scripts had to be done just by handwriting the code itself. We can do that by just clicking a plus sign and creating a shell of a message. I can then give that particular message context by choosing a specific target and an operation. So, in this case, we're going to choose the Salesforce.com operation. And it has 15-20 different operations -- 31 operations that it has inside of this particular web service.

Speaker
I counted them this morning.

Robert Holcomb - SOASTA
Okay, thanks. In this case, I want to log into the Salesforce.com service first. So, when I select that, you'll see that the message now has context, we know that it's a Salesforce.com web service based on the image that they are provided in and I also know that this is the login operation that will actually send my credentials to the Destination Web Service and return to me what they call a Session ID, which is really my authenticating token for my future messages.

So, instead of editing XML directly, which is what most people are doing today, I can they go into a graphical environment, what we're calling our Message Editor and graphically enter or enter my data in a graphic environment. So, in this case, I can enter my user name and password for Salesforce.com. Again, rather than editing an XML, I can do in this form, and the great thing about the project is I can execute this one message in this interface, just by pressing the 'Invoke' button or the 'Play' button down here and sending that message often to Destination Web Service and we get a response in a graphical, again a graphical format back from the Web Service.

Speaker
Can you open up that all the way?

Robert Holcomb - SOASTA
That indicates all of the fields that I've received back in the response that I sent to Salesforce.com.

Speaker
So, on the left there's what we sent and on the right is what you got back.

Robert Holcomb - SOASTA
So, this is an interesting case because the Salesforce.com service returns me this Session ID, which I need to use in subsequent web service calls. So, let's just go and save this message and I'm going to create what we call a 'Clip Property,' I may call that 'Session ID.'

Speaker
(Inaudible) around these big monitors is pretty difficult, its like falling around those little monitors.

Robert Holcomb - SOASTA
Well, I always complain the big monitors, yeah. Yours eyes end up doing a lot more moving, it gives you a little bit of a headache, so you can complain about even having a 30'' monitor I suppose But I've essentially created a variable that I'm going to hold back that Session ID that Salesforce returns. So, by going into the Message Editor, I can graphically attach the results of this message. Okay, so I can look at these results itself, and I can see these are not populated because I've not executed the message, but I could say, okay, so that Session ID I want to set to a property, and I'll see down below my list of all the different properties available that I've created. So, I've created a Session ID, where some system property is available as well. But the point here is that I want to be at least to attach this Session ID that's coming into the variable and put that in -- to take the results of that session ID put it into a variable.

So, I can again invoke this one particular message and I'll get the results back from that message and we'll get an indication that now this particular - we scroll down here -- we can look at this particular property has now been -- it was previously null, and now it has this particular Session ID. So, I can now use that in my future messages. So, that's a single message. So, in this particular Salesforce.com scenario, I want to make sure that this login occurs, before I go on to any other steps in my testing process, so I can add what we call a checkpoint and this checkpoint essentially says, execute every single one of the messages before this checkpoint, before we move forward. So, very important for dependencies to ensure that, that login occurs, because otherwise every other message is going to fail, because we don't have that Session ID back to us yet.

So, what I can do, is add another message and we would graphically attach a target in operation to this particular message, just like we have all of the login message. But what you'll see when we open this particular message in the Message Editor, is because this is a much more complex message than what the login message is. You'll see a lot of additional fields that obviously didn't exist in the previous message. So, we get all of that from the WSDL. In that we can provide this graphical form for you based purely on importing that WSDL into this product. So, Session ID in this case, same exact way that we populated data, form a response into a variable, we can put data from the variable into this message that we send to destination web service by clicking on the icon that brings up our click properties, and I can now -- this Session ID will now automatically be put in this message as we send it to this destination web service.

Speaker
Up-down the object types before you go further. I mean up in the -- on the right-hand side.

Robert Holcomb - SOASTA
Yeah, oh I understand. So, one of the things I've filled -- well, let me show you this first. So, the Salesforce.com service has these different 'S' objects that they call them, which are essentially creating all of the different objects inside of the Salesforce.com environment. So, all the cases, all the notes, all the accounts, which is what we're going to do in this particular demonstration, all can visually be created inside of this environment, and the other thing I want to show in this particular User Interface is that, if for example, you have a very large message you have to send and we've seen this for some of our early alpha customers, very large messages with lots of arrays. We can click simply on any number of different fields to narrow down what we're seeing on the right-hand side. So, managing very, very large messages become very simple inside of this particular environment.

So, in this case, let's just create a new account, we call it 'ABC Corp,' so, that's really I only need to do to create this message. I've attached in my Session Header, I've attached the Session ID property and I've given it a name. So, I can press 'Play' again for this particular message and again I'm sending a message now to the Salesforce.com service and what we'll see is we'll get a response and we can see that it is a successful response, the response is true. And we get the idea of what was created. So, that's great and the other thing I should note in the Message Editor is that we store an amazing amount of detail about this message, not only the soap messages that were sent and the property substitutions, the soap responses we've received, but also matrix around how long that message took to take, in this case it took to send. In this case, it looks like it's just under 400 milliseconds to send and get a response back from this particular web service. So, one of the things you have to do in testing all the time is validate that a particular message that's been sent, we get what we expect to we get valid response from that particular response. So, what we can do is, we can go over so success true, I want to make sure that every time I run this message it says that success is true. So, we go over here, we can look at the outputs and say, okay 'success,' we want that always to be 'true,' and we click the 'Stop' like icon to indicate that this should always say 'true.' And if we get anything that says, anything other than 'true,' then this validation is going to fail and we can then dictate how the test performs based on the response that we get back from that particular validation.

So, again I can directly just hit our 'Play' button here, just to test that validation. We can see that the response tells me that it has been a successful addition of any account. Then I can scroll down here and you'll see visually that, yes, that response did passed validation, in the case that it would not have passed validation, it'd be read and be very easy to detect what the problem was, and be able to solve that in the test which you're creating or solve the web service issue that generated that problem. So, this is the general process of creating individual messages within a clip. So, one of the things at this point -- I've created manual messages and you know what, this is certainly a lot easier than hand-editing XML inside of a Text Editor or writing a Full-Script to generate this stuff, but what we see this particular interface being really good at is being able to generate messages.

So, where we this going in the next couple of months is being able to auto-generate those messages for me. So, rather than me specifying, oh, I want ABC Corp, I want the user to define a set of rules, that will say, "Okay, I want a set of first names that are pulled from this SQL Database, I want set of last names that might come from a file with sitting down on a server." So, the point being that you can generate a set of rules that will automatically generate 10,000, 100,000 messages so that you can either load test your machine, your web service or go ahead and create specific functional tests based on rules that you have to find. So, we feel that, that's going to be very powerful addition to the product. What we could do today to generate a large number of messages is being able to duplicate messages. Okay, so I can go ahead and duplicate any number of messages and at that point if I want it, for in this case create.

So, I created 'ABC Corp,' I can go in and now create a very ingenious named company called 'DEF Corp,' if we want it to, just by simply opening up the message and changing that. Save that message, and back quickly, I'm able to take a message, duplicate it and configure it for potentially a different scenario that I want to create. I mean I've kept it very simple in this case. Obviously, if you're doing a real test, you would generate some additional data in each of these fields.

Speaker
Or you pull data out of a database to populate these fields or out of XMLs to populate those fields. You know the amount of variation that people want. What people really want to be to do is to be in an environment, where they're focussing on the task instead of the details of the task, and in this particular environment, the details go on forever.

Speaker
Oh, yeah.

Speaker
And so, you really -- our value proposition is to allow you to very, very quickly get started down a path where we provide value to you that first day. The testing we're showing here obviously your test scenario is going to be far more complicated than what we just showed you. But, the idea here is to put you in an environment where you can -- okay, so I'm not worried about semicolons; I'm not worried about the structure of the data. You trust us that we've gotten the structure of the data right, then you just have to fill in the scenario that you want in.

Speaker
So, we're almost out to take, so.

Robert Holcomb - SOASTA
Okay. Well, we'll make it pretty quick to wrap this up. So, this is just creating a specific unit test. Now, where really the power of the product comes in is being able to take all those individual unit tests and lay them out on a graphical timeline. Much like Ken was talking about in terms of creating home movies that I can visually take any number of different clips, in this case I'll just pull up a Salesforce.com clip we created. Just dragging that directly out of this interface, so I can pull those directly up and I create and layout a test however it is, however as a test designer want to layout a test. I can pull out as many of these guys as I want. In this case, these are all Salesforce.com clips, so I can pull out Google or Yahoo! or any type of Internal Web Service that you might have all an interact with all of them on this screen.

Speaker
What is the scale denoting then, the timeline?

Robert Holcomb - SOASTA
Perfect, so by default we're assuming seconds. So, we're sending essentially one message per second inside of each clip. By rescaling these message clips, I can now change how frequently I'm sending messages. So, rather than one per second which we default to I'm now sending in this case looks like about 6 per second based on the density dots that we've got inside here.

Speaker
So, then the scale will go all the way down to milliseconds. So, if you have something that really needs precision timing you can set it up or up to days. So, if you're running a...

Speaker
So, is this a way to do load testing?

Speaker
...we see this as you can do Functional Testing, Capacity Testing, Performance Testing, Stability Testing there's a 'Repeat' button in there, so I set up a test that runs for an hour and when it finishes it's started over again. And typically, you'll see servers -- server engineers want to do, I want to run for seven days without crashing and then we can ship. And so you just put it under a load and keep it under a load for seven days and then if there are no memory awakes, if they doesn't crash, something didn't break, then it's good to go out in this real world, and so the idea here is you can do little functional test then those objects can then be reused in order to create your load test as well, which would be a far more realistic load test than anything that you could do. So, you're going to play this.

Robert Holcomb - SOASTA
So, once you've laid out the test according to the timing that you want, in this case, I've laid down three different clips, which is pretty straightforward. I can press the play button and much like we did in playing a message now we're essentially playing out all of these unit tests on a line. I've configured that some of them are sending every 6 sec or 6 per second, some are doing one per second looks like one might even (Voice Overlap).

Speaker
Is it going to play?

Robert Holcomb - SOASTA
About five per second It's preparing.

Speaker
Okay, let's hope.

Speaker
It takes a little while there.

Speaker
No, it doesn't usually. So, I would say something is wrong.

Robert Holcomb - SOASTA
Usually takes about one or two seconds to prepare to play.

Speaker
Well, you know this was our high note, finish on a high note. You want to pull out that's pre-built.

Speaker
It is out for code.

Speaker
Yes it is, yes it is.

Speaker
And the demo guides...

Speaker
Yes, they always do.

Speaker
Doesn't happen just Bill Gates lays on stage.

Speaker
No, no, although, I've always felt really bad for the guy who built those demos for Bill when they didn't work. So, here you go, here.

Robert Holcomb - SOASTA
So, as the test now is playing out, you can see that the play head is moving across to indicate what is being sent at that given time. And you get a real time feedback about how long the message has been playing, how many messages has been successfully sent or received and we get feedback in the upper right-hand corner that indicates, yes this play was successful, we sent 71 messages, we've received responses to 71 messages and it took approximately 22 seconds to play that out. So, in a graphical environment we're able to keep track of real time what's going on with the test. So, we can see what's failing, what's succeeding and be able to alter this as necessary, so maybe I've learnt that the Salesforce.com clip should maybe started about five seconds later. So, I can just easily drag that, re-save and play it again.

Speaker
So, part of the idea here is that each of your unit test then become objects that you can very, very quickly drag and drop in order to create a new scenario. And then the reporting will come back to you basically by a clip or unit test, so you can then see, if in a really well-organized development team a large one. Okay, Joe your unit test broke -- here's what you need to do, here it is, reproduce it, get it fixed, get it migrated so we can restart through regression testing get this thing into production. Okay, so the idea is to go from all of this complexity immediately down to the detail and back and to provide the way to navigate that.

Speaker
Very cool.

Speaker
Yeah, thank you.

Speaker
Well, thanks for giving me a little tour.

Speaker
Yeah, my pleasure, pleasure to meet you.

Speaker
Thank you very much.

Speaker
All right.

Copyright ©2006 PodTech.net. All rights reserved. Privacy policy