The Australian Internet
Australian Cybermalls first
began trying to estimate the approximate size of the Australian Internet
in early 1996 when we commenced operations on the Internet. We do
this each month as a service to our visitors.
We reasoned at that time
- and still do - that a simple count of the number of registered domains
is an inadequate way to measure the true number of Australian
web sites, for the following reasons:
Not every registered domain
has a web site attached to it. Many companies register a domain purely
for email purposes. Many other domains have been registered by companies
who plan to use them at some future time, but have yet to do so.
Many web sites - especially
in 1996 and 1997 - exist under subdomains or on subdirectories
of their ISP's or host's web servers. In the earliest days of the Net
in Australia, domains weren't seen as particularly important by many legitimate
Not every Australian web
site has an .AU domain (eg: Telstra.com, BigPond.com)
Many domains are multiple
registrations owned by the same company which point to the same web
site (eg: a manufacturer may register a domain for each of its major
brand names or trademarks; a newspaper proprietor may register a domain for
each publication in its stable)
Many domains have been registered
for purely speculative purposes. Their owners have no intention of
ever attaching a web site to them and are simply hoping to be bought out
(eg: aol.com.au was originally registered by a speculator, and was subsequently
acquired by AOL when they opened operations in Australia).
Because most .AU domain
registrations last for two years and the failure rate of web site operators
is notoriously high, many "dead" domains exist in the official registry
at any time. They aren't weeded out of the registry until the owners fail
to renew them.
Instead, we reasoned that any
legitimate and serious web site would list itself in the major
search engines and would include at least some geographic information in
its text (eg: the site operator's office address).
Furthermore - since the
overwhelming majority of Australia's population live in the country's capital
cities - we reasoned that by searching spider-driven engines for these capital
cities we could derive a rough approximate of the actual number of truly
"live" Australian sites, regardless of what domain they operated under.
We recognise that this is
a far from perfect methodology. But in essence, it's very similar
to the well-known ANZ Jobs Index (which attempts to estimate trends in employment
by counting the number of job advertisements that appear in capital city
This may also explain why the
estimated number of live sites we report is only about 33% to
40% of the number of domain registrations that exist at any time.
Changes In The
During the period between 1996
and 2000 we've been forced to change the search engines we poll
to derive the AIGI several times. This has sometimes resulted in unexpected
upward "blips" in the numbers of sites we report.
The reason for this is that
although we always poll multiple engines to derive an overall average figure
for each capital city, we found that changes in the way the engines operated
during this period necessitated dumping some and moving to others as time
One serious issue that
emerged during 2000 was that we noticed that many former "name" search engines
appear to have simply stopped adding new sites to their databases.
Our monthly polling clearly
showed us that these engines were returning the same numbers to our queries
from one month to the next. Our own site statistics confirmed that these
engines were no longer sending out spiders for indexing purposes.
This discovery tends to confirm
a widely-held suspicion amongst some Internet marketers that many of the
earliest Net search engines have either "maxed out" their capacity and/or
have effectively been abandoned by their operators, though they still give
all outward appearances of being live.
This discovery has important
implications for both web site owners and search engine users.
For web site owners, it may
indicate that their ability to gain exposure in some very well-known search
engines is now compromised (ie the engines still accept listings,
but do not actually add those listings to their files).
For search engine users, it
may indicate that some major search engines are now seriously out
of date and are becoming increasingly irrelevant as true guides
to the Net from one month to the next.
Naturally, we welcome any
correspondence on the
AIGI - though we hope this answers the most common questions you might
have about it.