I constantly hear people say, oh just Google it – Google has everything. I love Google and use it every single day, but Google does not have everything. You ask, how can that be. Here’s why. Google and other databases manage, index, and deliver results from what’s called the Surface Web. The Surface Web includes links, information, reports, news, subject matter data, and of course advertising data that funds Google and makes search “free.” But the Surface Web accounts for only a small fraction of the content available on the Internet. It is estimated that the Surface Web has about 20 terabytes of information, or between 5-10 percent of the information available for open search.
So what about the remaining 90%? The remaining 90% of web content is called the Deep Web, Invisible Web, or Hidden Web. Don’t confuse the Deep Web with the Dark Web. They are different animals. Content on the Deep Web is content hidden from Google and other search engines because to access it requires a login, password, a subscription, direct access, or an account to query it. There are thousands and thousands of databases, files, and pay-walled content that can’t be reached from the Surface Web. For example, let’s say you want to get the 10k report for Amazon or Google or some other publicly traded company. There are several places you can go to retrieve this information, but the Securities and Exchange Commission (SEC) allows anyone to search, download, and print the 10k reports for publicly traded companies. However, to get this information, you will need to go the SEC website and look for Edgar (Electronic Data Gathering, Analysis, and Retrieval system) to find the 10k report for any publicly traded company. Google can bring you to the front door so to speak by getting you to the SEC – EDGAR page but it can’t search EDGAR because it can’t query the EDGAR database.
As I said, there are thousands and thousands of databases that are available, but not through Google or any other Surface Web search engine. So if you are looking for business/industry content, historical and archival information, government statistics, legislation, court opinions, regulatory information, weather and crop information, academic and scholarly works, government procurement opportunities, medical and scientific papers, clinical trials, and COVID-19 data you can get to the site via Google but you will have to learn the search interface to query the database and extract the information you need to answer questions, complete your analysis, and deliver your evidence-based report.
As an information professional, I have access to many proprietary/paywalled/subscription databases that have content inaccessible via Google and I know how to use them. Each database or proprietary subscription has its own unique interface, features, and search tools. Most companies don’t have access to proprietary databases which can cost a lot of money and most companies do not have staff trained to use these databases.
Searching for information is easy, finding the information you need to solve problems and answer questions is hard. A trained information professional will save you time, money, and deliver actionable intelligence that will facilitate and enable you and your team to make the right decisions based on fact-based content.
