Transcript: Google's Shashi Seth has your custom search engine

This transcript is from a PodTech.net podcast at:
http://www.podtech.net/home/technology/1670/googles-shashi-seth-has-your-custom-search-engine

Guest: Shashi Seth - Google
Host: Robert Scoble - ScobleShow

Robert Scoble - ScobleShow
So, who're you? It's my favorite question, famous starting question.

Shashi Seth - Google
I'm Shashi Seth, came to Google about 19 months back.

Robert Scoble - ScobleShow
So, we're at Google?

Shashi Seth - Google
We're at Google.

Robert Scoble - ScobleShow
We're not at some other company.

Shashi Seth - Google
Yeah, we're at Google.

Robert Scoble - ScobleShow
That's why you're wearing a Google shirt.

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
I'm wearing my FireFox shirt.

Shashi Seth - Google
Awesome, my favorite browser. So, I think...

Robert Scoble - ScobleShow
So, what do you do here?

Shashi Seth - Google
I'm a Product Manager. My responsibility is in the areas of personalization and community efforts and search. My team focuses on products like iGoogle, personalized homepage.

Robert Scoble - ScobleShow
I've never even heard of iGoogle.

Shashi Seth - Google
You've never heard of iGoogle? Personalized homepage?

Robert Scoble - ScobleShow
Oh does that the personalized -- I see -- I don't use that.

Shashi Seth - Google
Okay.

Robert Scoble - ScobleShow
I got so many things to learn here.

Shashi Seth - Google
We got to convert you over. (Voice Overlap)

Robert Scoble - ScobleShow
Well, see, I'm making a tour. I went to the Google Reader team and they converted me to Google Reader and that's awesome by the way. So, I'm making my tour... (Voice Overlap)

Shashi Seth - Google
Well that's the next product we need to sell you and it is the personalized homepage. So, you can get a personalized experience out of whatever gadgets you want to on your homepage and there are tons of these gadgets available; anything from stock quotes (ph) to DIG modules to real time flight information, whatever you want is available and you can add them, change them around, flip them around, you can have multiple tabs.

Robert Scoble - ScobleShow
See, I liked it. I don't call it iGoogle because it, in my referral log its just says "IG".

Shashi Seth - Google
"IG", ig, that's how we refer too it internally. It's "IG."

Robert Scoble - ScobleShow
I'm going, - where's all the traffic coming from? It's IG, because once in a while, my name shows up in a headline on some gadget there, a whole lot of traffic will come from IG.

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
And I am like, "What's iGoogle? I've never heard of that."

Shashi Seth - Google
It's quickly becoming one of our most popular products. So, my team focuses on all personalization efforts, whether it's implicit personalization or explicit personalization. We've tons of products in that space.

Robert Scoble - ScobleShow
People are waving at me from the window. You guys can't see that at home, but there's a lot of Google offices right here.

Shashi Seth - Google
We have alerts in there, we have personalized search in there, Google Co-op and so on and personally I'm responsible for Google Co-op. One of the latest products we launched here is Google Custom Search Engine.

Robert Scoble - ScobleShow
Okay. Is that what you call Co-op?

Shashi Seth - Google
Co-op is a suite of products and it has three products in it. The way we segment Co-op is how you customize Google.com and how do you customize your own Website with a search engine that is flavored or is meant to cater to the audience and the content on your Website.

Robert Scoble - ScobleShow
Got it. And so I think that's where I keyed in on you, as somebody said, "You got to see the Custom Search Engine which is, why we are here. So, what do I do with this Custom Search Engine?

Shashi Seth - Google
So, imagine you run a blog, right and you want people to be able to not only search over your blogs, which is easy enough, but also say, you want them to search over pretty much anything in your blogroll. You want, maybe, all the URLs that you've ever referred to in your blogs to be searchable as well. So, if somebody types in say, "Browsers," they should be able to pick out any URLs that refer over to a browser company, a new plug-in, whatever it might be, and just be able to search about that. So, the goal is that...

Robert Scoble - ScobleShow
So, wait a second, so this Custom Search Engine can track what URLs I'm linking to?

Shashi Seth - Google
If you plug into our API, it can. Otherwise you can just explicitly go in and tell us to do that by adding URL sites, URL patterns whatever you want. So, let me give you a simple example, so imagine you're on an automobile site and you have a search engine there and you don't want the user to have an experience that when they enter the word 'Jaguar' to be finding Jaguar, the animal and not Jaguar, the cars. You also want your users to find some really good content, that you think is good first and then be able to find other content as well. You can actually go in and take a slice of our index, define how the ranking works in the search engine, so that your content shows up higher and leave it at that. So that when users come they can say, "I prefer, say, results from xyz site first, then I'd like my partner sites to show up, second and then, maybe all of the Google.com shows up rest ," or you want to say, "No I want a very defined experience, I only want you to search over these five sites because I think that the content in these sites is really good."

Robert Scoble - ScobleShow
And like that I can limit the search resulted to whatever URLs I put in there?

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
Can I upload OPML file? How do I?

Shashi Seth - Google
Yes, you can upload an OPML file.

Robert Scoble - ScobleShow
Well that's sexy. So, Google understands OPML now.

Shashi Seth - Google
Yes, we do.

Robert Scoble - ScobleShow
Yeah. So, I can -- my blogroll available as an OPML. In fact, I think Google Reader lets me export OPML.

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
So, 350 feeds that I'm reading...

Shashi Seth - Google
Take your OPML, have the Custom Search Engine just suck it in and that's the start of your index, so to speak. Once you start with the index, then there's a lot of other things you can do. You can choose to re-rank the results, You can change the look and feel to match your content, your Website. You can invite your community to participate in it as well.

So, you can open it up to, say your editors and have them do certain things, like tell us what new URLs they should be adding in there, maybe even tag the URLs, so that you can have refinements in there. So, when somebody enters, say the word 'Canon D50 camera,' they get a range of options that they can click on and say, "I'm looking for manufacturer pages of Canon D50," in that way, it narrows down the search results for you and gives the users a very precise amount of information that they're looking for.

Robert Scoble - ScobleShow
Okay. What's the best use of the Customized Search Engine that you...?

Shashi Seth - Google
We constantly get surprised, by how people use it; there is not one that stands out, there is just range of things that people have done with this. Some really good examples are, Intuit has launched a site called JumpUp.com; which has meant for its users to find small business information on the Web. They create a lot of content themselves, but they also know that there's a lots of great content out there for small business owners, like say, small business administration, or financing in various ways and so on and so forth.

What they've done is on their site, they provided a search box that gives you two options: search JumpUp.com or search the Web, which has related or relevant content related to your query. So, if you type in something like 'small business loans,' you're going to get very different results from that then you would get on Google.com. And the Editors of JumpUp.com have rated that content and said, "Yup, this is really good content that our users should be able to find." Now, some of those may have been pretty arcane and may have not shown up on Google.com per se, but if the editors believe then that is good content for the users, so be it.

Robert Scoble - ScobleShow
Can I do searches based on time? For instance, kind of see all the news about Procter & Gamble in the last hour or the last two hours.

Shashi Seth - Google
So that's not built into the system per se. There isn't a flag that you can turn on and say "sort by time," however, you can tag the URLs in your index appropriately to say, "This is in the last day", "this is in the last hour," and so on. So, that people can get those refinements at the top of the page to say, "I'm searching for Canon D50 say blogs, I want them in the last hour, I want them in the last week, last month," and so on. And if the user clicks on any of those options then you narrowed down the search results. What that requires you to do is, to really put in the time and effort to go and tag each URL or URL patterns with that kind of tag or label as we call it.

Robert Scoble - ScobleShow
Can I search by language?

Shashi Seth - Google
Absolutely.

Robert Scoble - ScobleShow
If I have Chinese Website and I only want Chinese...?

Shashi Seth - Google
That comes out of the box, also...

Robert Scoble - ScobleShow
So, we were talking about Chinese language.

Shashi Seth - Google
Language support comes out of the box much like Google.com, if you go and type in a different language, we'll try and return results in that language. It doesn't matter where that search block lives. So, exactly the same thing happens in Customized Search Engine, if you have a search engine that starts out with English and if somebody enters in and say, Portuguese characters in there, then you're going to get back results in Portuguese. So, that's turn on out of the box. There's a really good example of that, there's a Website that the United Nations has called Food and Agricultural Organization, Fao.org. And if you go to their Website, they have a custom search box on their site and it is built with the idea that if you enter queries in Spanish, French, Italian et cetera, the results would show up in those languages.

Robert Scoble - ScobleShow
Right. Can I do you searches just on one of the specific search engines like image search, or on a video, or...

Shashi Seth - Google
So, currently Custom Search Engine out of the box is built to support Web results. However, we do have an integration with our AJAX search API that allows you to actually get custom search results for different indexes or different corpora. So, you can get blogs, you can get video, you can get images and so on and so forth. Also gives you the flexibility of rearranging the results in very different ways than what Google.com does and get a very different experience. We've a couple of people who use this; the only limitation that is there, is that results are limited to eight results at a time through this API.

Robert Scoble - ScobleShow
How do I -- where do I learn, where do I go to learn about this?

Shashi Seth - Google
Code.Google.com.

Robert Scoble - ScobleShow
Code.Google.com.

Shashi Seth - Google
Yes. This is where all our APIs, external facing APIs live on Google and that's the perfect place to start.

Robert Scoble - ScobleShow
Okay. How hard is it to you in -- so I have a Weblog or I've a company Website, how hard is it to build this thing?

Shashi Seth - Google
Using the AJAX search or just the Custom Search Engine?

Robert Scoble - ScobleShow
Custom Search Engine.

Shashi Seth - Google
I'm willing to bet that you should be up and running in 30 minutes or less. If you can export your OPML file, import it into Custom Search Engine, that's a really great start. Within five minutes you have an index, another 20 minutes spent tweaking the look and feel and the ranking; if you want to change the ranking and you're up and running. We give you a code snippet, HTML code snippet that you cut and paste, put it on your Website and you're up and running.

Robert Scoble - ScobleShow
Okay. Are there any tricks that you would tell somebody who's doing this for the first time, or things to think about or ways of searching that it get a better result for your users?

Shashi Seth - Google
Sure. I think there are three things to really think about. One is, do you want to provide the users with a way to narrow down the search results, the refinement things that I've talked about earlier. It's a very powerful concept; we have it on Google.com for health and automobiles and so on and so forth. So, it gives you a narrow slice of the vertical by looking to a query and saying, You know you just asked for, say Hepatitis, maybe you're looking for more than Hepatitis, right. Why don't we offer options like clinical trials, symptoms, treatments and so on and so forth?

Robert Scoble - ScobleShow
(Voice Overlap) that's what you see at the top (Voice Overlap)

Shashi Seth - Google
Exactly, refine your results by. That exact same facility is provided in Custom Search Engine as well. You have the ability to define what those treatments are, what those refinements are, and then you can go and tag your URLs in your index with those labels, so that it narrows down the search results.

Robert Scoble - ScobleShow
So, I could create my own refinements because often times I find the refinements haven't got into it the way I wanted to do.

Shashi Seth - Google
Yeah. You're in total control with those refinements.

Robert Scoble - ScobleShow
How do I do that on -- what do I have to do to tell the engine I want, like if I'm searching for HDTV I want to put manufacturer...?

Shashi Seth - Google
Sure, it's very simple. Our philosophy behind Custom Search Engine was that we didn't want people to really have to know how to program. So, we provided a bunch of different user interfaces that once you create your slice of the index, you can go in and tell us, whether you want refinements, what those refinements are going to look and feel like? Should there be some background kinds of things that you need to do. So for example, do you want to add synonyms to these labels, so that you just add those synonyms to the query terms and just get different results out of it. You can also then use those refinements and attach labels to the URLs in your index, so that next time the search is done, and somebody clicks on, say, "Clinical Trials", then all the clinical trial labeled URLs will flow up to the top.

So, it's very simple, very straightforward, no programming required, you can just do this in a matter of a few minutes. As long as, you can think through the taxonomy or the words that you want to use for the refinements, I think the rest is simple.

Robert Scoble - ScobleShow
Okay. Anything else, I need to know about custom search?

Shashi Seth - Google
There were two other things that I wanted to pitch. One was the ability to re-rank, which is, we talked about earlier, which is...

Robert Scoble - ScobleShow
Well you don't rank me on top?

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
Hey I don't know. If you do a search for "Robert" for instance, I'm on top, really Robert Kennedy, or Robert De Niro, or Robert...

Shashi Seth - Google
So, imagine that you get to play around with our ranking and say, "I like this result to be higher," or "This set of results to be lower," and so on, you can tweak the ranking to get to the point, where you're comfortable with the user experience. The third thing is the ability to create Oneboxes. I'm not sure if you're familiar with Oneboxes that we show up on Google.com, but let's say, you type in a query like -- let's pick a current movie Casino Royale, James Bond and you type it into Google, you're going to get a result at the top that actually gives you an icon of a film reel and next to it, it says "Casino Royale, 2 hours and 17 minutes, here are the characters who're playing in it," and so on, and it gives you a link to go book that movie if you want to.

So, imagine you can actually create that same effect in your Custom Search Engine, not just movies, whatever you want. You get to define when does this Onebox occur and what kind of information will it show?

Robert Scoble - ScobleShow
So, if I do a search for like lets say, "Cool Web 2.0 sites," I can put whatever I want to be in that box.

Shashi Seth - Google
Exactly.

Robert Scoble - ScobleShow
How do I do that?

Shashi Seth - Google
There is an XML interface for you to define the queries on which it should trigger and what should this look like, what we give you is control over four lines of text and URLs and then image and you get to control what shows up, when. So, you can have multiple instances of, for these queries, I want this kind of result to show up, and for these queries, I want this kind result to show up. And that's an XML file that lives on your Website and you get to change it as often as you want. So, you can change it every minute, if the information is real time, you can change it once a day, whatever you're comfortable with and it just shows up.

Robert Scoble - ScobleShow
Very cool. Anything else? You did was that number two?

Shashi Seth - Google
That was number three. (Voice Overlap).

Robert Scoble - ScobleShow
Do you have some demos?

Shashi Seth - Google
I do. I do want to show you a couple of really cool Websites that have implemented this technology. There're thousands of these out there, so...

Robert Scoble - ScobleShow
Don't build that if you don't get (ph) shown off.

Shashi Seth - Google
I'm just amazed by how innovative people are with this stuff and what kind of things they're doing with it.

Robert Scoble - ScobleShow
Okay.

Shashi Seth - Google
The first one I'm going to show you is Macworld. This is the Macworld site. Notice in the top of right corner they have a search box which says "Google Custom Search" this is using Google Custom Search, focusing on that and type in, say a query like "iPod".

Robert Scoble - ScobleShow
I'm a little too close.

Shashi Seth - Google
Should I move this further away?

Robert Scoble - ScobleShow
Yeah. That's fine.

Shashi Seth - Google
Okay. So I entered the query "iPod", and I go in for a search and you'll notice that the whole experience is on Macworld itself, nothing has changed. We're still on Macworld, but they've search results showing up and these search results are very, very different than what you would see, say on Google.com. Notice that if you went to Google.com, iPod is likely to generate an Apple.com site, but Macworld chose not to have that and have a bunch of other sites here. They also have refinements here, so if I'm looking for iPod news, it should allow me to narrow down the search results for iPod news and that's what it is. You've got a bunch news sites, news information about iPods shows up here.

Robert Scoble - ScobleShow
And that limited to just (ph) Macworld site.

Shashi Seth - Google
Not just Macworld, they have a bunch of other partners. Macworld is part of the IDG group and therefore they want content from all the IDG partners showing up here as well, and that's what they've essentially done.

Robert Scoble - ScobleShow
Wow!

Shashi Seth - Google
Let's take a look at another interesting Website called JumpUp.com. JumpUp.com is an Intuit Website as I'd talked about earlier, which is focused entirely on small business content. Notice that they've done a slightly different treatment with their search box, where they have search JumpUp.com or search business resources on the Web. If I click on that, that takes me to Google Custom Search right there. Let's enter a query like say "Small Business."
If I could only spell and you're going to get results that Intuit has chosen to be appropriate.

Robert Scoble - ScobleShow
Now the ads here -- because I saw on both these sites they have Google Ads, do those sites ask for Google Ads or do they get them automatically?

Shashi Seth - Google
So, out of the box you get ads built into this system since custom search engine is free of cost, the Google Ads are required, are the only people that we exclude ads are for are non-profit institutions, government institutions and educational institutions. Everybody else has to show ads, except if you are an AdSense partner or you become an AdSense partner, we split the revenues with you. So, you also make money from these ads as well.

Robert Scoble - ScobleShow
So, you should have also sign up for AdSense at the same time you're signing up for a Custom Search Engine.

Shashi Seth - Google
Actually Custom Search Engine has that whole experience built into it itself, so if you are an existing AdSense partner, it can give you the ability to give us your AdSense account number, if you are not one, it actually gives you the ability to signup for one.

Robert Scoble - ScobleShow
Okay, great. And any others?

Shashi Seth - Google
If you are interested I can show you a couple of screen shots for Google Custom Search and I'll show you how easy it is to build one. So, if you go to google.com/co-op, you're going to see "Create your own search engine", no worries. Right at the top, if you click on it, that's Custom Search Engine. Let me sign-in, to show you an example of a Wine Search Engine that I had built for myself. So, I'm going to show you one that I've already created, called 'My search engines,' I click on that and I get to the Wine Search Engine that I had created. If I click on the control panel, it shows you all the things that I had done here. So, right at the top, you'll see that there are things like what's in the index, what kind of refinements you have, Look and feel and so on, and each of these are basically a bunch of forms that you fill out and interact with, to make your search engine work better.

So, really no programming required here, take a look at all the search engines, all the URLs and sites that I have in here, the sites that I want excluded are in here as well. So, you get to both include and exclude sites, so if you don't want your competitions showing up here, you actually can just tell us, "Hey exclude my competitions site from here". You get to create refinements, pretty simple straightforward method, you'll see, I've a bunch of refinements here like reviews and points and so on and so forth. You get to change the Look and feel. So I had chosen a Look and feel that you can see right here, it's a pretty funky color, but matches my blog site about wine. You get to cut and paste the code directly from here, so all you have to do is, just copy this code, copy it, paste it on to your blog HTML and you're done.

There's a Collaboration suite here that allows you to invite other people to contribute to your search engine. Here's the Make money part, which is linking your AdSense account or creating a new AdSense account into Google Custom Search and there's an Advanced section that does require a little bit of programming and the Advanced section allows you to do the ranking changes for the Custom Search Engine. And finally, you get a homepage or a Preview page that allows you to play with this. So if I say enter a wine say Duckhorn and I say Cabernet, then I'm going to get results that are meant specifically for that.

So you'll see right here I've results for Duckhorn Cabernet 750 ml. and you get the price, so these are very different results than you would have ever gotten from Google.com because I know a little bit about wine and I can restrict my wine search over a certain set of URLs or sites, gives me very different and very relevant and contextual results than Google would have given normally.

Robert Scoble - ScobleShow
Very cool.

Shashi Seth - Google
Thank you.

Robert Scoble - ScobleShow
Anything else that we need to talk about...

Shashi Seth - Google
No, this is great.

Robert Scoble - ScobleShow
Because this is awesome stuff.

Shashi Seth - GoogleBowel sounds are present.
Thank you.

Robert Scoble - ScobleShow
I am all (Inaudible) to go and try this on my blog.

Shashi Seth - Google
Awesome.

Robert Scoble - ScobleShow
I just interviewed Mark Lucovsky he showed me the APIs too so.

Shashi Seth - Google
Right.

Robert Scoble - ScobleShow
And all sorts of stuff to do, and all sorts of home work projects for the (Voice Overlap)

Shashi Seth - Google
Exactly. I know you'll be up tonight doing the stuff.

Robert Scoble - ScobleShow
Thank you very much.

Shashi Seth - Google
Thank you, pleasure meeting you.