Yahoo!/AllTheWeb's image search syntax

No one lights a candle and hides it under a bushel, except Yahoo!

First published @ Searchlores in June 2007 | Version 0.01 | By Nemo

Introduction

AllTheWeb's syntax

References

Introduction

Is AllTheWeb dead? Not quite true, AllTheWeb image search (the engine powering Yahoo! image search) is alive and kicking and has undergone major redevelopment last year. Unfortunately Yahoo seems to be shy and once more seekers had to join the dots to guess the syntax, which is the most advanced on earth (followed quite closely by exalead, who offers regular expression searches). The purpose of this essay consists precisely in documenting Yahoo's image search syntax and providing examples showing its usefulness.

To begin, how do we know the syntax? Well, Yahoo bought the three Google's competitors -AllTheWeb, AltaVista and Inktomi- in 2003/2004, so it is a good guess that one of these is powering Yahoo image search. To know which one, all we have to do is testing the three search syntaxes:

AllTheWeb: url.tld:edu, url.all:gif & filesize:43
AltaVista: domain:edu (3,559,461 hits), edu (3,559,461 hits), "philosopher's" NEAR stone (7 hits) & "philosopher's stone" (12,617 hits)
Inktomi: domain:edu (3,559,461 hits), edu (3,559,461 hits) & originurlextension:gif

As we can see, AllTheWeb search syntax, cf. [1] and [2], is the supported one. On the other hand, Paul Bausch published on Yahoo! hacks the Yahoo video search syntax, which already have leaked to the web, cf. [3] and [4]. As Yahoo video search also supports the AllTheWeb search syntax, it is a good guess that image and video search should share the same query language, which is the case for most operators. Seeing the names of the operators, another one was guessed: fromurl:.

Now that you know how I have found the syntax, lets see how to use Yahoo image search to its full power.

AllTheWeb's syntax

Elementary seeking-jutsu

Lets start by the most basic seeking-jutsu -the image's color- which is useful for searching old images, when photos where black & white. Compare: Champs-Elysées (black & white) vs. Champs-Elysées (color). Or for searching scans, as documents are usually black & white. Compare: Ode to Joy (black & white) vs. Ode to Joy (color).

Another easy one is image's format (JPG, GIF or PNG). Almost everyone knows that GIF and PNG are great for computer generated graphics, clip art and drawn graphics with few colors, or large blocks of color, whereas JPG is far better for photographs and illustrations with large numbers of colors, as these formats give better quality images for the same file size. As far as I know Yahoo image search do not have a file type operator, the best we can do is searching for keywords in the url and the operator is url.all:. Compare for instance:

Intermediate seeking-jutsu

Like it or not, you must use words for searching images and, besides, images search engines are totally blind! For instance, if you are searching for oranges, Yahoo! shows you oranges not because it is able to identify images' content, but because the word oranges appears, at least, in one of the following locations:

image's file name,
image's alt attribute (an HTML snippet of text which is supposed to describe image's content to blind people using an audio or braille browser),
a small snippet of text in image's neighborhood,
image's and document's URL, perhaps document's title and anchors (link's visible text) of links pointing to the document containing the image.

To get a better grasp of what the hell I'm talking about, take a look the same query, oranges, at AltaVista image search, which is powered by AllTheWeb image search. If you click on the link More info of the fourth image search result, you'll see the small snippet of text (Abstract:), taken from the original document, describing the image.

Now that you grasp the importance of names, lets see how AllTheWeb handle words. AllTheWeb uses an automatic Boolean AND between terms, but it also offers a way of searching for any of your search terms (OR operation). Compare illumination parchment and (illumination parchment). The main uses of the OR operator are the expansion of you search query with synonyms, because different people use different words to describe the same thing, or to give a context to your query. For instance, if you were searching for illuminations on parchment, you could use the following expanded query: illumination (parchment vellum), where the expanded terms are substituted by the synonyms enclosed by parenthesis. To get synonyms, you can use the following online tools:

Warning! For reasons that will be explained later, as the respective syntax, images from the site flickr.com will be excluded preemptively joining -fromsite:flickr.com to the following queries.

Lets see another use of the OR operator -giving a context to your search query-. If you where searching for illustrations of Captain Nemo -fromsite:flickr.com or Gandalf -fromsite:flickr.com, you would be disappointed with what you get, as search results are flooded with commercial trash and littered with a bunch of pictures of someone's favorite pet. The root of the problem is that search engines do not have a clue of what do you want. It's up to you, to give them the context of your query string. That is fairly easy to do, just go to the Seekers' Oracle, type two keywords who belong to the same context in the box, select web search, hit search and you'll get a list of keywords, sorted by frequency, who appear in Yahoo's web search results and are likely to be related with your search terms. The rationale is that is far easier to get related keywords by seeing them, than by brainstorming to get them. With this in mind, lets see how do we give a context to our two queries. For the first, using the keywords nemo verne we easily spot the following keywords belonging to our context –verne, jules, captain, leagues, sea, under, nautilus, mysterious, submarine, island, underwater, ocean, voyage & crew– which allow us to build the following query:

nemo (verne jules captain leagues sea under nautilus mysterious submarine island underwater ocean voyage crew) -fromsite:flickr.com

Similarly for Gandalf, using the keywords gandalf tolkien, gandalf aragorn and gandalf frodo we manage to give a context to our query:

gandalf (tolkien rings lord hobbit wizard ring frodo middle earth lotr bilbo fellowship grey white aragorn king return towers balrog saruman hobbits galadriel legolas sam baggins sauron battle elrond gimli gollum magic moria council theonering gwaihir elves hobbiton pippin three wise battles thorin evil shire arwen merry boromir gondor theoden tirith faramir eomer minas elf rohan secret eowyn son heir isengard rivendell denethor army arathorn palantir samwise throne pipe wood quest journey mordor rivendell friends light orcs gamgee destroy sword dead death precious mithril) -fromsite:flickr.com

But only a quarter of the job is already done, as till now we only have eliminated the documents where only appear our main keyword, either by design (off topic pages or pages containing random lists of keywords made by incompetent spammers) or by chance (someone's pet name). Of course savvy SEOs also have done this kind of research and have stuffed and optimized their pages with keywords from this context... Notwithstanding they are betrayed by their own pages, which expose the keywords showing the ways they are using to monetize their "content"... The way of handling these offenders is getting those very keywords. For that, we once more use the Seekers' Oracle, but this time we use only one keyword to maximize the crap to content ratio on search results! As web and images are likely to be monetized in different ways, we choose the option images to spot the keywords clinging to our main keywords on images search results. For instance, using the keyword nemo we spot the following list of troublesome keywords: finding, disney, movies, movie, shop & jamba. All we have to do is excluding the pages containing those very same keywords:

nemo (verne jules captain leagues sea under nautilus mysterious submarine island underwater ocean voyage crew) -fromsite:flickr.com -"finding nemo" -disney -movies -movie -shop -jamba

Similarly, using the keyword gandalf, we spot the following list of troublesome keywords –movie, movieshots, products, product, shop, banner, movies, cinematrix, game & games–, which allow us to build the following query:

gandalf (tolkien rings lord hobbit wizard ring frodo middle earth lotr bilbo fellowship grey white aragorn king return towers balrog saruman hobbits galadriel legolas sam baggins sauron battle elrond gimli gollum magic moria council theonering gwaihir elves hobbiton pippin three wise battles thorin evil shire arwen merry boromir gondor theoden tirith faramir eomer minas elf rohan secret eowyn son heir isengard rivendell denethor army arathorn palantir samwise throne pipe wood quest journey mordor rivendell friends light orcs gamgee destroy sword dead eowyn death precious mithril) -fromsite:flickr.com -movie -movieshots -products -product -shop -banner -movies -cinematrix -game -games

Some quarries are better described by phrases. The "philosopher's stone" is one of those examples (phrases are searched by enclosing them between "). To get more search results, lets search for the philosopher's stone in different languages. Thanks to Wikipedia, thats a trivial task. Take a look at the following page: Philosopher's stone, where you can get the translation in a bunch of other languages, just following the links present on the table In other languages located on the left. So our quest for the philosopher's stone is expanded by the following query:

("Philosopher's stone" "Философски камък" "Pedra filosofal" "De vises sten" "Stein der Weisen" "Piedra filosofal" "Ŝtono de la saĝuloj" "Pierre philosophale" "Pietra filosofale" "אבן החכמים" "Filosofinis akmuo" "Bölcsek köve" "Философски камен" "Steen der wijzen" "賢者の石" "De vises stein" "Dei vises stein" "Kamień filozoficzny" "Философский камень" "Kameň mudrcov" "Viisasten kivi" "Bato ng pilosopo" "賢者之石")

Using the Seekers' Oracle to investigate the phrase philosopher's stone, we get the following list of off topic keywords –harry, potter, dvd, youngactors, harrypotter, movie, shop, product, products, hogwarts, fawkes, games, game, trailer, amazon, cd & shopping–, which we promptly use to refine our query:

("Philosopher's stone" "Философски камък" "Pedra filosofal" "De vises sten" "Stein der Weisen" "Piedra filosofal" "Ŝtono de la saĝuloj" "Pierre philosophale" "Pietra filosofale" "אבן החכמים" "Filosofinis akmuo" "Bölcsek köve" "Философски камен" "Steen der wijzen" "賢者の石" "De vises stein" "Dei vises stein" "Kamień filozoficzny" "Философский камень" "Kameň mudrcov" "Viisasten kivi" "Bato ng pilosopo" "賢者之石") -harry -potter -dvd -youngactors -harrypotter -movie -shop -product -products -hogwarts -fawkes -games -game -trailer -amazon -cd -shopping

Until now we only have half refined our three main queries: nemo, gandalf & philosopher's stone. The other half requires advanced seeking-jutsu and those techniques will be taught in the corresponding section.

AllTheWeb also offers powerful ways of refining our queries by document's / image's location. There are two prime cases where location is very useful. When you want to explore the content of a specific site:

site:searchlores.org –images hosted at searchlores.org site (operator used site: aka url.host:),
fromsite:searchlores.org –images from searchlores.org site plus external linked / hotlinked images (operator used fromsite:),
fromsite:searchlores.org -site:searchlores.org –external images linked / hotlinked by searchlores.org site,
site:searchlores.org url.all:images –images from searchlores.org site located at the directory images (the operator url.all: is used to search keywords anywhere in image's URL),
site:searchlores.org fromurl:realicra –internal and external images referenced by pages from searchlores.org site located in the directory realicra (the operator fromurl: is used to search keywords anywhere in document's URL, not necessarily in image's URL),

or when you want to search some classes of sites / documents having better signal to noise ratios:

Images on educational sites. If you are searching for images which might interest academicians, you can search for images referenced by educational sites. Examples: (fromtld:edu url.host:ac url.host:edu) bestiary -fromurl:wjh or (fromtld:edu url.host:ac url.host:edu) kells. The operator fromtld: is used to search for images referenced on a page at a top-level domain, the operator tld: aka url.tld: is used to search for images located at top-level domain and the operator url.host is used to search for keywords somewhere in the host name.
Public domain images. You can search public domain images, for instance, at the Project Gutenberg site. Examples: site:gutenberg.org tale or site:gutenberg.org fontaine. Oh boy, I almost have forgotten the wolf and the fox in the well fable hehehe.
Free images repositories, which are mainly used on fora and blogs which do not host images. These repositories are useful, if you want images which might interest those who post on fora or write blogs. Example: (site:imageshack.us site:photobucket.com) (naruto uzumaki).
Images not located at flickr.com, as that site has been recently flooded by normal users, which, as you can guess, usually only have normal tastes. Untill Yahoo! comes with a way of processing efficiently this very low grade ore and they know they have a problem, because images from flickr.com have been in and out of the index the last days, we'll take care of the situation resorting to use -fromsite:flickr.com.
Open directories. Seekers love to search things on open directories, because they get less SEOed / filtered search results. Apache open directories are easy to find, because their URL often ends with strings like '?N=D', '?N=A', etc.. Examples: vermeer (fromurl:"n=a" fromurl:"n=d" fromurl:"s=a" fromurl:"s=d" fromurl:"d=a" fromurl:"d=d" fromurl:"c=d" fromurl:"c=m" fromurl:"c=n" fromurl:"c=s" fromurl:"o=a" fromurl:"o=d") or escher (fromurl:"n=a" fromurl:"n=d" fromurl:"s=a" fromurl:"s=d" fromurl:"d=a" fromurl:"d=d" fromurl:"c=d" fromurl:"c=m" fromurl:"c=n" fromurl:"c=s" fromurl:"o=a" fromurl:"o=d").
Images located at fora / messageboards. If you are searching for good fora / messageboards, images offers a good way of seeing what they worth. Almost everyone who runs a forum / messageboard uses a prebuilt software from a small set of common ones. As a consequence, almost every forum has one of the following keywords in the URL: viewforum, read, topic, viewpro, viewthread, threadid, showthread & forum. So if you were searching for French stock market fora, you could use the following query: "cac 40" (fromurl:viewforum fromurl:read fromurl:topic fromurl:viewpro fromurl:viewthread fromurl:threadid fromurl:showthread fromurl:forum).

Advanced seeking-jutsu

AllTheWeb offers five powerful operators which allows you to focus on images' properties and not on the surrounding context:

aspect: search for images with a specific aspect ratio (the image's width multiplied by 100, then divided by the height and rounded down, if necessary). The aspect ratio has an important feature, it does not change when the image is rescaled (unless the image becomes too small that we start to have rounding problems). Can be exact number or range (use <, >, and [ ; ]).
date: search for images that have been last modified on a certain date. You need to use a Unix-style date, which is the number of seconds elapsed since January 1, 1970 (Unix timestamp). You can use the following tool to convert dates to a Unix timestamp. Can be exact number or range (use <, >, and [ ; ]).
filesize: search for images with a specific size in bytes. Can be exact number or range (use <, >, and [ ; ]).
height: search for images with a specific height in pixels. Can be exact number or range (use <, >, and [ ; ]).
width: search for images with a specific width in pixels. Can be exact number or range (use <, >, and [ ; ]).

Month:
Convert a Date/Time to a Unix timestamp (number of seconds elapsed since midnight UTC of January 1, 1970)
	/	/	at	:	:

To get a better grasp on how these operators work, lets start with the date: operator. The date: operator has two main uses: seeing how something or someone has evolved with time, fravia for instance, taking one year time slices (or any other time span you prefer), to do that we enclose the beginning and ending dates between square brackets

1999 fravia date:[915148800;946684800]
2000 fravia date:[946684800;978307200]
2001 fravia date:[978307200;1009843200]
2002 fravia date:[1009843200;1041379200]

2003 fravia date:[1041379200;1072915200]
2004 fravia date:[1072915200;1104537600]
2005 fravia date:[1104537600;1136073600]
2006 fravia date:[1136073600;1167609600]

or seeing how an important event changed someone / something, for instance the World Trade Center before (on the left) and after (on the right) September 11, 2001

world trade center date:<1000166400

world trade center date:>1000166400

Before we give the final refinements to our queries (nemo, gandalf & philosopher's stone), lets solve three of fravia's riddles, which are quite instructive in the way they illustrate how to use the other four operators.

Riddle I (from An ochre colored "image searching" riddle): Searching an image without knowing its name. Can you locate/understand/explain this image?

Right clicking on the image, we know that image's width = 691 pixels, height = 460 pixels and file size = 42162 bytes. This data is usually sufficient to uniquely identify an image on the web and allows us to write the following query – width:691 height:460 filesize:42162 –. There is only one search result and is located on this forum. I guess this a copy from somewhere else and the image's name (probulemfrog.jpg) should be the original one. Searching probulemfrog on Google, we finally locate the original site.

Riddle II (from Hidden essays): Well, ahem... on a real ~S~ site a small (sortof) "riddle" is imho necessary and welcome (per angusta ad augusta). sincerely hope you'll have quite a difficult task :-) trying to fetch my own "dark" and "light" essays...
~S~ fravia+, September 2000

Right clicking on the images, we see that both have width = 301 pixels, height = 366 pixels and their names are mucha3.jpg and mucha4.jpg. Seeing this information, I guess that the "hidden" page has two mucha images and those images probably have the same width and height. Testing that hypothesis (width:301 height:366), we find on page 20 (ymmv) two other mucha images located here. The caption of the second image on that page, the width and height of the images?, is quite interesting... Was this kind of search, what fravia had in mind?

Riddle III (from not so easy little historical 'riddle'): For those that like riddles (and web-searching for images). How comes there's a swastika (green, smack in the middle, see?) on this 1917 russian 1000 rubles banknote? And: can you find the other side (the one with the building) on the web?

Right clicking on the image we see that its width = 818 and its height = 495 pixels. We also know that this image is a scan of a real note with unknown width and height, but the ratio width / height (aspect ratio) of an object remains unchanged by magnifications or reductions, so the original banknote has an aspect ratio of approximately 165... giving some room to the way the images on the web might have been cropped, we guess the aspect ratio of those images should range from 155 to 175. Joining all the information, we construct the following query: aspect:[155;175] (1000 rubles banknote) 1917.

This query can be expanded in other languages using the Wikipedia and following the links present on the table In other languages located on the left. For the word ruble we get its plural in other languages either by consulting the tables or images captions or banknotes' values followed by something. For the word banknote, we get the translations seeing the title. Taking this linguistic expansion, our query becomes:

aspect:[155;175] (1000 rubles rublů Rubel rubla rublos roubles ruskih rubli roebel рублей рубаља rubalja ruplaa Rubler карбованців 卢布 รัสเซีย Banknote "Gîn-phiò" Банкнота Bankovka Monbileto اسکناس Billet Nota 지폐 שטר כסף Banconota Popieriniai pinigai Bankbiljet 紙幣 Pengeseddel پۇل Banknot Банкнота Bankovka Bankovec Seteli Sedel ธนบัตร Банкнота 紙幣) 1917

Among these pages, this one contains the other side of the banknote, cf. other side 1 and other side 2.

Finally, lets see how these operators can be used to further reduce the noise in our search results:

aspect: this operator is very handy for three tasks:
1. getting ride of images which are too wide or too tall, because they are almost always buttons and banners (aspect:[30;300]),
2. getting ride of photos (no more photos of someone's pet or some crap on sale :), thanks to the fact that there are two main standards for photos aspect ratios: 4 × 3 or 3 × 2 (-aspect:[66;67] -aspect:75 -aspect:133 -aspect:150),
3. getting ride of images of movies, as those images have aspect ratio of 4 × 3 or 16 × 9 (wide screen) (-aspect:133 -aspect:177),
width: this operator provides a handy way of trashing small images. It is up to you, what is too small (I become more picky the more search results I get :).
filesize: provides a handy way of getting ride of light images which often have poor quality. As a rule of thumb, I usually chop the 10% lighter ones.

It's now time to give the final refinements to our three queries (if you get We did not find results for... hit the button reload, as we are pushing AllTheWeb to the limit! Trust me, AllTheWeb can handle it :)

nemo (verne jules captain leagues sea under nautilus mysterious submarine island underwater ocean voyage crew) -fromsite:flickr.com -"finding nemo" -disney -movies -movie -shop -jamba aspect:[30;300] -aspect:[66;67] -aspect:75 -aspect:133 -aspect:150 -aspect:177 width:>250 filesize:>10000

gandalf (tolkien rings lord hobbit wizard ring frodo middle earth lotr bilbo fellowship grey white aragorn king return towers balrog saruman hobbits galadriel legolas sam baggins sauron battle elrond gimli gollum magic moria council theonering gwaihir elves hobbiton pippin three wise battles thorin evil shire arwen merry boromir gondor theoden tirith faramir eomer minas elf rohan secret eowyn son heir isengard rivendell denethor army arathorn palantir samwise throne pipe wood quest journey mordor rivendell friends light orcs gamgee destroy sword dead eowyn death precious mithril) -fromsite:flickr.com -movie -movieshots -products -product -shop -banner -movies -cinematrix -game -games aspect:[30;300] -aspect:[66;67] -aspect:75 -aspect:133 -aspect:150 -aspect:177 width:>500 filesize:>50000

("Philosopher's stone" "Философски камък" "Pedra filosofal" "De vises sten" "Stein der Weisen" "Piedra filosofal" "Ŝtono de la saĝuloj" "Pierre philosophale" "Pietra filosofale" "אבן החכמים" "Filosofinis akmuo" "Bölcsek köve" "Философски камен" "Steen der wijzen" "賢者の石" "De vises stein" "Dei vises stein" "Kamień filozoficzny" "Философский камень" "Kameň mudrcov" "Viisasten kivi" "Bato ng pilosopo" "賢者之石") -harry -potter -dvd -youngactors -harrypotter -movie -shop -product -products -hogwarts -fawkes -games -game -trailer -amazon -cd -shopping aspect:[30;300] -aspect:[66;67] -aspect:75 -aspect:133 -aspect:150 -aspect:177 width:>200 filesize:>10000

References: