[ But first this: I know, I know, you
nearly choked on your cornflakes when you saw another newsletter,
blog type... er... thing - less than two weeks since the last
one. But this is a special Christmas edition with an exclusive
interview with Craig Nevill-Manning, Senior Research Scientist
with Google. It's a little present from me and Christine.
Merry Christmas! ]
~ * ~
I fly a lot. I mean a LOT. So I'm pretty
much used to walking around airports without my shoes on and
with my laptop floating away on a tray with the laptop bag
behind it. I get "selected" randomly in various
airports for those special little checks from time-to-time.
I don't mind. It makes me feel all the more secure when I
know that these kinds of precautions are happening.
But nothing could have prepared me for
the little episode that took place as I left the Search Engine
Strategies conference in Chicago last week.
Originally, I was due to fly directly
to Chicago from London and then return. However, it was decided
that I was required at Boston HQ (home of iProspect). Not
that I was complaining much - it was the staff party last
Friday night. And who'd want to miss a night at The Armani
Cafe on Boston's famous Newbury Street? (They even have their
own "wee-fee" network - Italian designer "wee-fee"
Anyway, I go to check-in at the United
desk - only to remember that we hadn't cancelled the UK flight.
So, according to the global system, there I was, missing an
international flight, taking an internal flight, and with
a one-way ticket booked to the UK from another airport!
Now, if you were security and you noticed
that - and also saw me with a cardboard box wrapped up with
Duck tape... What would you think?
Yep! And so did they. Boy was I searched
good and proper.
In fact, I believe that those guys in
O'Hare airport probably know me as intimately as my wife and
Not all of it was sooo bad though...
So, to business. The conference last
week was huge. It vastly exceeded the numbers expected. So
the walk-up sales must have been pretty high. I got to meet
fellow Brit Andy Beal of the hugely successful search marketing
firm KeywordRanking. And that deserves a mention only because,
we've been passing each other in corridors at these events
without ever knowing we had fish and chips and lots of rain
in common. 'Nice bloke' as we say in the UK.
As ever the sessions were very lively
over the entire three days. I did my usual "organic"
session with my team- mate Brett Tabke from the world's leading
search forum, WebMasterWorld. And yes, the first topic was...
I think the walk-up sales at the conference
may definitely have been as a result of that last update and
people just trying to find out whatever they could.
Anyway, I then joined my friend Paul
Gardi (SVP Search Jeeves/Teoma) and Marissa Mayer (Director
Consumer Products, Google) for our "Link Building"
I had a feeling that Marissa might have
been swamped with questions about Florida. Yet, even though
the topic was broached, it was all very contained. Not so
in the first session though. This started with one guy who
appeared to want to form a lynching party and head right up
to Google's HQ at Mountain View and torch the place!
I'm sorry, but I still maintain that
these conspiracy theories about Google are nothing but poo!
So, having arranged a timely interview with Craig Nevill-Manning,
Senior Research Scientist at Google (and inventor of Froogle),
I asked the BIG question. And Craig, bless him (and he is
such a great guy) was happy to oblige with a full answer.
Just before we get to the interview,
I want to ask you for your attention and endurance just a
little longer. In the last epistle (if you were inflicted
with it), I wrote a little about information retrieval and
how it differs from pure data retrieval or information management.
I mention it again, because it's important
to understand that a ranking algorithm is based on more than
computer science and statistics. It's ridiculous to imagine
that someone would dominate the results at a search engine
interface, purely because they had more words, or more pages,
or simply better HTML with words stuck in better places. If
that were the case, my son, who's an excellent coder, but
knows nothing about nuclear physics, could knock out a 100
or so pages cut and pasted from some books and be number one
for a subject he knows nothing about, and has no authority
in the field.
First generation search which was based
purely on text and (mainly) the vector space model, proved
that it could be so easily manipulated and that this older
technology (for it is that) really couldn't scale with the
web. We simply spend too much time thinking about text and
contextual analysis to remember that, the web is, very much,
an example of a social network. Social network theory is concerned
with properties related to connectivity and distances in graphs.
For instance, networks of social interaction are formed between
academics by co-authoring research work. In that example,
citation analysis provides a way of identifying central or
Another area within the science is known
as 'sociocentric network analysis.' This examines complete
networks of actors and relations. Here, the study is of global
properties of a network and characterizing the position of
any given actor by reference to all the others (think link
equity - think "what are you known for?").
Okay, so you can hear the propeller on
my head starting to whirl around again. But I just want to
help you think a little more about how the web, in the way
that we link to each other, and communities interact with
each other reflects what happens in real life. A ranking algorithm
needs to incorporate this 'relationship analysis' as well
as understanding the 'textual matrix' between terms and pages,
in order to return authoritative documents as indicated by
the community: Not the owner of the search engine.
It's a fascinating science information
retrieval on the web. And the more you know about what the
scientists at search engines are trying to achieve in this
unification of text and link analysis - the less frustrating
it is when you see a major change take place such as the Florida
episode of last month. And you'll see other nuances at the
interface which compensate at times.
While all eyes were on ranking reports
and much hair was being pulled, discretely, Google introduced
Froogle into the main body of results for certain queries.
So the many conspiracy theorists are
incorrect when they say that Google has dumped the most popular
'commerce' queries from the top of the results to force people
into buying AdWords. There ARE results at the top of the page
AND they are FREE listings AND they are pure COMMERCE listings.
Does that mean that last month wasn't
such a big deal? Not at all. I believe it's all part of the
ongoing experiment to provide authoritative listings - separated
from the pure commerce listings - separated from the advertising.
It's very much those invisible tabs that Danny Sullivan has
so succinctly put his finger on.
Did it work? I don't think so. Will we
see more changes? We certainly will. Read the interview with
Craig Nevill- Manning and find out for yourself where Froogle
fits in the long term... And how, as he says himself about
Florida: "We don't claim that we're perfect either! You
know, we rolled out a change in the search results... We're
going to tweak that and change it and get feedback from people
and improve it."
So, there's a fact: Everyone is allowed
to screw-up sometimes and be forgiven - even Google.
~ * ~
And that, for me, is the end of a brief
era of shoddy timing and relentless non adherence to the rules
of email marketing.
Next year I'm back with stabilisers!
Or maybe I should just say the incredible "adept at everything
search engines" Christine Churchill. (For those who care
: She did let me build a little quirkiness into the recipe.
I must still be around in the new year after all ;-)
Read on, dear reader - read on!
CHICAGO SEMPO MEETING:
AN INSIDERS PERSPECTIVE
Christine Churchill has a quick perspective
on the proceedings of the SEMPO meeting in Chicago. Hear it
from the "horse's mouth"... What... WHAT??? I know
you love horses but I was just being metaphorical...
<< Play >>
One of the landmark developments in the
SEM industry this year was the formation of a professional
search engine marketing association called SEMPO < www.sempo.org
Now SEMPO is one of those subjects that
elicits strong emotions from people - they either love the
idea or they want to crucify you for being associated with
it. As a member of the Board of Directors, I've had many "discussions"
about the pros and cons of SEMPO. Relax guys, it's just an
organization. It can be anything you want it to be. I'm not
here to debate whether we need such a group or not, the organization
is here - it's real - there are over 120 members and sponsors
like Yahoo/Overture and Google are throwing money at it. It's
SEMPO had its inaugural meeting in San
Jose in August 2003. At that meeting we launched the organization's
new web site and opened the doors for membership.
The day before the Chicago SEMPO meeting,
the Board sat down in Noel McMichael's Marketleap suite (which
I noticed had a well stocked bar) and we went over the meeting
agenda. Since the meeting the next day didn't go exactly as
planned (more on that in a minute), in my mind there were
two meetings - the planned meeting and the real one.
The Chicago meeting was to be both a
"here's what SEMPO has done since August," "a
presentation on the results of the SEMPO member survey"
and a quick blurb on "here's some plans for next year."
The meeting was also seen as a chance to talk directly with
SEMPO members and solicit feedback on a variety of subjects.
As I mentioned earlier, the actual SEMPO
meeting didn't exactly go as planned, but what does in life?
We can laugh at it now, but the Chicago SEMPO meeting had
a sudden change of events when minutes into the opening presentation
a disgruntled Labor Union representative turned off and then
removed the projector.
Undaunted by the now infamous "projector
episode," SEMPO's enthusiastic cheerleader, I mean President,
Barbara Coll (aka WebMama) continued on unfazed (okay a little
fazed, but she covered it up well). Barbara verbally described
recent SEMPO activities and new web site additions (new ROI
based case studies, numerous new search engine marketing articles,
and an excellent SEM glossary edited by Danny Sullivan himself
along with generous help from Kevin Lee and a team of volunteers).
Barbara also announced the addition of the new Job Board that
was launched only days before the meeting. Who needs a PowerPoint
presentation anyway? (Note to self - next meeting bring a
Yes, the informational meeting was also
a recruitment effort. We recognize that for the organization
to be strong it needs to be representative of the industry
and the only way we can do that is to have a large membership
base. Barbara encouraged companies on the fence to go ahead
and join now by leaking the news of an upcoming membership
price increase. Join now or pay more later! (Note: Apparently
the recruiting call worked. We are thrilled to announce that
15 new companies have signed up in the week since the meeting.)
If you'd like to see the presentation
Barbara was going to present, you can download the PowerPoint
version at: < http://www.sempo.org/downloads/Chicago-SES-2003.ppt
Special thanks goes to Net Applications:
> for sponsoring an open bar that had brisk business during
the SEMPO meeting. Thanks guys - you did your part in helping
things stay cool.
The original agenda for the member meeting
had Noel McMichael (SEMPO VP and CEO of Marketleap) give a
presentation on the results of the recent SEMPO membership
survey. The idea behind the survey was to solicit information
from SEMPO members including what they wanted the organization
to do and their reasons for joining. Again due to the "projector
episode" this became the "radio version" only.
If you'd like to view the presentation on the survey results,
Noel's slides are posted on the SEMPO web site.
One interesting item that the survey
documented was the motivation behind members joining SEMPO.
It wasn't to get freebies, discounts, or links, but rather
most members joined:
1.) To raise awareness of SEM to the
general marketing community (70% of respondents)
2.) To influence industry direction and
3.) To get a stronger voice in dealing
with search engines(36%)
4.) To network (after all, we are a social
The overall message from the Meeting
was "SEMPO works for the membership." One thing
I will say about SEMPO management - they are an open minded
and accessible group.
If you have ideas on what the organization
should be doing, talk to us. I promise you none of us bite
(okay, well not very hard and we've had our rabies shots.)
Okay, enough of the SEMPO meeting. Oh
- the Overture and GoToast parties after the meeting were
a big hit and for some, who shall remain nameless (but who
has the initials M.G.), the parties lasted until 4 AM. Back
to you Mike...
[ Ed: At 4.00am I was simply delivering
newspapers, as I do at that time - this is where the confusion
arises I think... Anybody??? ]
(c) Christine Churchill 2003 - Net Writer
FROOGLE AND THAT
I had the pleasure of working with Google's
Senior Research Scientist, Craig Nevill-Manning in London
earlier this year. So it was great to know that I'd be seeing
him in Chicago for a "catch-up".
Craig's a Kiwi (from New Zealand) and
not only is he one of the smartest scientists in this entire
industry; he's one of the nicest guys you could wish to meet.
He's based in New York, is a bit of Knicks
fan and partial to a bit of sailing around the Statue of Liberty!
Originally, we were due to do an "In-Conversation"
piece about Froogle generally. But of course, in between,
we'd had that Florida update and the discrete debut of Froogle
listings at the top of the results pages.
Find out everything you ever wanted to
know about Froogle. And then get the answer to the BIG question:
"What happened last month Craig?"
<< Play >>
It's always nice to get to know some
background Craig. I know a little, but in your own words,
where are you from and... How did you get here?
Okay, so, I did my PhD in computer science
in New Zealand. And then I came to the US for a post at Stanford.
I then decided to take up a faculty position, so I was Professor
at Rutgers, which is the State university of New Jersey.
Actually, Stanford is where I first met
Larry [Page]and Sergey [Brin], who, as you know, went on to
found Google. Then six months after I moved to New York, they
emailed me and asked me if I wanted to join them!
I said: "No, no I'm doing the professor,
academic thing..." Anyway, you make these decisions in
life! And so, a couple of years later, I finally changed my
mind and moved out to California and joined Google.
I'd done some research stuff on the academic
side with information retrieval, and then at Google I worked
on a whole bunch of things, including Froogle.
I came up with the idea, prototyped it
and then led the engineering team that put it together.
But I know from when we met last in London,
that you're actually based in New York. Right?
Right. So about six months ago I moved
out to New York to start up Google's first satellite engineering
office. Essentially, we figured we'd hired all the good programmers
in Silicon Valley... So we're now going to hire all the good
ones in and around New York. [laughs]
So, does that mean that Froogle is now
based in new York?
No, it's not actually. The Froogle engineering
team is still based back in Mountain View. So I spend a lot
of time on planes...
So, tell me: What are your overall responsibilities
with Google right now?
Well, it's a combination. I'm managing
Froogle... Well, everyone else does most of the work these
days. But I still make sure it's headed in the right direction.
And, as I said, I'm also hiring and managing people for the
New York office.
Okay: Froogle. Was that your idea? Was
it a kind of innovation? Was it something the R&D guys
at Google were already working on...
It was, kind of, partly me and partly
the idea of product manager Pearl [Tsai] Renaker who'd already
kicked the thing off. The observation was, that, sure, Google's
great for many things. But if you're trying to buy something
Google doesn't always have the coverage. And especially the
user interface when people are trying to buy. When people
are buying things they want to see photo's - they want to
see prices called out etc. That's something that Google.com
doesn't do. So Froogle is trying to fill that niche.
And over time, what we're going to do
is try and integrate it more and more. Essentially to improve
Google search results for these kind of product queries.
I'd like to come back to the technical
side a bit more in a second. But, is it safe to say that,
a lot of what we can find in Froogle is made easier because
it's more difficult to crawl that kind of shopping, commerce
Yes, yes. So, one of the issues is definitely
crawling. Shopping sites are much more difficult to crawl
than the average site. I know you're familiar with session
ID's and so on, which get in the way and give crawlers fits...
You know, we've been working on some
techniques to crawl better. And I guess the other aspect from
a technical point of view is coming at from information extraction.
That is, you've got a page on a shopping site and it has five
prices on it and six photo's etc. etc. How do you identify
the correct price and the correct image... And so it goes,
the correct name the correct description, the correct category...
And that's really a problem of extracting structured information
that is fielded like database information, from, well a big
chunk of HTML.
So, in very simple terms, for the average
end user: What's the difference between a query for 'digital
camera' as a general search over at Google - as opposed to
a search for 'digital camera' over at Froogle?
That's a good question! In one? The results
you get look a lot different. We have the image called out,
we have the price called out. The user interface is different
so you can restrict by price, and sort by price...
All of those types of features are additional,
on top of what Google normally offers. And the other thing
is that, Froogle's pre-identified a sub-set of the web which
is about products. So if you know you're interested in buying
a product - you go to Froogle!
But, if you're kind of doing research,
if you want a review or you want the manufacturers site -
you go to Google. So Froogle's really the last point in the
cycle where, you know what you want to buy and you just want
to find where to buy it.
Yeah. I've done a few selective searches
recently on specific items and I seem to be seeing Froogle
results at the head of the page. I mean it looks as though
those results are being returned by Froogle. So does that
mean Froogle's out of beta. Is Froogle now live?
It's not out of beta yet. There's still
a lot of work to Be done. It's now at the point, we think,
where it's useful to people. We're VERY conservative about
bringing things out of beta. But, essentially, it shows up
for a number of searches, where... Well, where we think people
are basically looking to buy something.
How's it going to be presented? I mean,
will Froogle have it's own interface or will there be a combination
of general and shopping results or...
[Laughs] GOOD question! We're working
our way around that right now. I mean, Google does a LOT of
experiments. You know we try to figure out...
[Bursts out laughing] Tell me about doing
experiments after what happened last month! [referring to
the Florida update]
Yeah... alright... exactly...
Mike, we're continually working to improve
peoples' search experience. That's what keeps people coming
back and clicking on ads and... So, we're making sure that
when people are looking for things to buy we can simply give
them the information they want.
We'll see how this experiment goes, we'll
probably modify things in the future...
From an awareness point of view, do you
think the user knows that these results at the top of the
page are not paid for? You know like AdWords or sponsored
You know, really I don't know. They're
not paid for. And it's a little bit difficult to communicate
that to users. Maybe they do look a little more like ads because
they're actual products that people are trying to sell.
Maybe we do probably need to work on
that and how to communicate that to the users. BUT - the bottom
line is - they are unpaid listings, they're unbiased. They're
the best results we can find for those products online.
Okay. So, what about submitting to Froogle
as opposed to Google? I've noticed that there are already
people creating little software packages, little utilities
as Froogle feeders. Are these sort of things supported by
Well we don't support them - but we encourage
them. Let me put it that way. There's a distinction there.
You know, Froogle accepts feeds...
Sorry to interrupt Craig, can I just
keep it fairly simple for the average user here. If I'm a
guy with a smaller site, with not too many products...I know
about submitting to Google and how if I'm linked I'll get
crawled and stuff... But how do I get my stuff into Froogle?
The easiest way is to put together a
text file... the feed format is a tab separated text file,
which you can export from Excel, for example. And in that,
on each line, you put the name, the price, the URL for the
image, a URL for the page itself - the destination page -
and a description of the product and a category.
Most of this information the merchants
already have. And it's just a very simple tab separated text
format. They upload that to our FTP server. You set up an
account in advance so that can get access, and then you can
update that as often as you like.
That kind of information is already available
if you go to froogle.google.com?
Yes, that's right. If you go to http://froogle.com/merchants.html
there's a whole description of how to proceed.
Just from an online marketing point of
view, one of the biggest areas, next to search is affiliate
marketing. It seems to me that this could hit some of those
affiliate guys pretty hard.
Why do you think that?
Well, with affiliate marketers taking
a commission on every sale they make on behalf of the merchant
and they're making their own affiliate sites to promote on
the Google.com interface... If I'm the owner of the affiliates
program, I just feed my stuff into Froogle and then I don't
have to pay anyone commission...
Yeah. Although there is still some extent
to which the affiliate sites form a useful function. In some
cases there'll be a site where the owner hasn't bothered to
put together a Froogle feed themselves. And in those cases
we'll sometimes get feeds from affiliates instead.
You still end up at the merchants site.
But because they haven't bothered to put a feed of their own
together, there's a possibility for someone to fill that gap.
So that brings us to an existing problem,
where sometimes, again on the main results page, you end up
with the affiliates turning up in the results more often than
the owner of the affiliate program. What would happen if you
end up with the top ten results at Froogle coming from ten
affiliates of the same product?
We work pretty hard to avoid that. When
we accept a feed, we want to know that it's not going to just
duplicate a feed we already have, for example.
And we also crawl sites. So if we've
already extracted information from a merchant we're not going
to accept an affiliate feed for the same data.
So, you're right, in that case the affiliate's
sort of shut out. If we already have the data - we don't want
duplicate information - as you say Mike.
So what about the future then? It's not
quite out of beta, but it's kind of there. Will there be a
big "roll-out" with a fanfare and that sort of thing?
Weelll... Overtime we're going to give
it more and more exposure. Obviously around holiday shopping
season people are searching more, so we wanted to give a little
more exposure by making it appear for a certain subset of
It's really a judgement call for us about:
"Has it reached a level of quality where we actually
think it enhances the user experience?"
We believe that's the case at this point
- but there's a lot of work still to be done. You know, part
of the increase in quality will come from the merchants getting
involved. There's kind of a catch 22 here. Merchants are not
going to give us a feed until they know it's going to pay
off for the traffic. Part of the reason for us making it a
bit more visible is to say to merchants: Look we're serious
about this. Give us your feed. Costs you nothing - put in
a little effort and you'll get some free traffic from us.
So it is completely free. Is there likely
to be a time where you'll charge?
No. It'll be free forever. We've learned
from Google.com that if you provide a great service then you
can derive income from putting ads down the side. And we're
going to do exactly the same... Well, we're already doing
the same for Froogle.
On a general level Google.com competes
with MSN and Yahoo! and Jeeves etc. But from the Froogle side,
from the shopping bot and comparison shopping side, there
are not that many players, but it is a whole different set
of competitors you have out there...
Yep, yeah. It's a very competitive area
simply because there's money changing hands when people shop...
But it's a lot of fun. From a technical point of view it's
a good challenge...
Craig: There's one question I really
have to ask you right now. [Laughs] What on earth happened
last month? [referring to the Florida update]
[Huge laughter] You mean with the Google.com
main results - yes? Mmmmm...
Craig, I get asked this question by everybody.
And I don't want to give you a hard time. But... You know,
there are conspiracy theories and... Well it would be lovely
just to be able to say something more than my own idea...
Right, right... I understand. There's
no conspiracy. This was simply an effort to make our results
better on Google.
You know, at Google we have this Church
State wall between the people who sell the ads and engineers
that work on making the Google search engine better.
We determined, for certain kinds of queries,
the search results had... Well, many irrelevant results. We've
tried to weed those out by changing our algorithm.
So, whenever we change our algorithm
it's going to hurt some people and help some others. So it's
a tricky business. But the bottom line for us is: "Is
it going to be better for our end users?"
And... We don't claim that we're perfect
either! You know, we rolled out a change in the search results...
We're going to tweak that and change it and get feedback from
people and improve it.
So, there are going to be a number of
iterations and people are going to see more changes...
Undoubtedly! As it always has been. This
may be a larger change than many changes that have happened
in the past. But the bottom line is the engineers that have
worked on that, and I know them very well, so I know their
motivation, and they're totally focused on making the top
ten results as relevant as possible. So, if you were inside
Google, you'd know that these conspiracy theories just hold
You know, I've tried to explain to so
many people at this conference that there is a sort of "Chinese
Wall" or however you want to refer to it inside Google.
But there are still so many guys convinced that there is a
conspiracy and all of the previous results for the most popular
keywords have been buried in place of education sites and
government sites, so now everybody has to open an AdWords
account... It doesn't make any sense to me I have to say...
Well now you can use Froogle as a counter
argument to that. Product related searches... You can see
we're putting free listings out for people who have submitted
to Froogle. So, thinking about, that could potentially drive
away AdWords traffic. So, you'd need to come out with a consistent
conspiracy theory about all of these things...
Craig - I think you've been grilled enough
[Laughs]. And I know you have a conference session to attend.
You've been great thanks so much for giving me your time.
And thanks to you Mike.
(c) 2003 Net Writer Publishing.
LITTLE THINGS WORTH A MENTION. Includes:
o WHAT YOU NEED IS... SEARCH MARKETING
ADVISOR: It's big AND it is clever! And it features real word
online marketing case studies and news from experts such as
the great man himself, Fredrick Marckini. (And I may even
beg to be in it myself one day!)
o LARRY CHASE'S ESSENTIAL SEARCH ENGINE
MARKETING RESOURCE GUIDE: Special edition. At no cost to you,
get SEM info from industry leaders and an eccentric Englishman
called Mike Grehan!
o BRYAN AND JEFFREY EISENBERG WANT ME
TO MENTION THEIR NEW REPORT: These are the FutureNow Inc.
guys from New York. They have persuasion... And a huge amount
of architecture between them... Could I refuse? Not likely...
o ASK CHRISTINE: Here's your opportunity
to get some very valuable and free advice from a leading search
engine marketer. Yes, our very own Christine presents an excellent
site clinic session at the SES conferences - and now she's
bringing it to our happy little e-rag. Find the email address
below to get Christine to take a quick peek at your web site
and give a little friendly critique.
o MIKE, THAT HUBS AND AUTHORITIES AND
COMMUNITIES BOLLOCKS YOU'RE ALWAYS GOING ON ABOUT: I don't
understand it all! Then you need my free paper on the subject
dear witless one! Seriously, for those who missed the opportunity
of the free paper at SES Chicago, you can get a link below
to: A look under the hood: Link equity explained.
o WEBMASTERWORLD.COM ANNOUNCES PUBCONFERENCE
V1 IN FLORIDA: I'll be there with my SES team-mate Brett Tabke
- find out more!
Editor: Mike Grehan. Search
engine marketing consultant, speaker and author.
Associate Editor: Christine
e-marketing-news is published
selectively on a when it's
At no cost you may use the
content of this newsletter on
your own site, providing you display it in its entirety
(no cutting) with due credits and place a link to: