"DuckDuckGo, Startpage and Qwant are Microsoft and Google's bedfellows. Not their competition... Brave Search will be competition - not a bedfellow."
In the last week, Brave - the browser provider - announced a buyout of a low-profile search engine called Tailcat, and a plan to integrate it into the Brave brand. We might expect that to raise a few groans of: “Oh please, not another 'private search engine'”, but this one genuinely does look different, and I think it's a really exciting development.
[UPDATE: July 2021. You're reading a speculative post based on Brave's pre-launch outline of the product. Now that Brave Search has launched in beta, I've posted a detailed assessment of its reality. You should also note that at the time of this update, Brave Search resides on an Amazon server.]
In the past I've babbled on at length about supposedly “private search engines”, their suspiciousness, and the glaring contradiction of them interfacing with megatrackers' search results, serving megatrackers' ads, etc.
I admittedly do use DuckDuckGo as my main 'superficial' search engine at present. Not because I think it's inherently private. Or because I necessarily believe it's much more than a Microsoft partner. I mean, run a TraceRoute on DDG and you'll see the access path invariably goes to Microsoft and then blocks visibility. So no, I don't trust DDG. I use it because I can reliably run it via Tor, which means that whatever it's doing behind the scenes, it's not getting my identity [UPDATE: this is not always true, as DuckDuckGo is involved in running Tor]. I can't use Google via Tor. If I could I would.
The core problem with the currently significant “private search engines” is that, to coin the phrase I used in 2018, they're all “in bed with the enemy” - locked into deals with oppressive trackers and wholly dependent upon them for survival. This is where Brave Search is set to change the landscape.
Tailcat - the product on which it will be based - queries an independent index and does not rely on trackers for its results. Indeed, Tailcat's developers are known to be unconvinced by the supposed privacy offered by tracker-fuelled “private search engines”. Josep Pujol - Tailcat project lead - has gone as far as to describe the ongoing development as:
“the only real private search/browser alternative to Big Tech available on the market.”
Reading between the lines, in my interpretation at least, that's an insinuation that “private search engines” like DuckDuckGo, Startpage and Qwant are not really private.
NITTY-GRITTY
Of course, having a private search index is one thing, and having a private search index that can compete with Google is very much another.
Brave has stated that the quality of the index will be refined by humans, in the form of “anonymized contributions from the community”. This was a system I predicted would come to the fore some years ago. In the 2015 piece Google Search Now Has Limited Lifespan, I said…
“Using millions of human beings to verbally recommend and point, rather than a mechanical computer code à la Google, makes more sense as a means for people to find their way round the Web in the future... I strongly feel that the advancements in search relevance are going to come through the addition of human input features.”
However, building full ranking integrity with this method is going to take time, and there will likely be problems relating to really obscure content. Only by crawling the entire Web the way Googlebot does, and then offering highly advanced search features, can obscure material be made adequately accessible. A community can't rank what it can't find.
My expectation is that Brave Search, upon launch, will be good with populist searches, but will not for a very long time rival Google for the really deep dig. In fairness, the vast majority of searches people run are superficial, populist and relating to the same set of trend-led topics and names. So a comparatively small but common-needs-focused index would still work for most people. But the real game-changing factor will lie in the effect a genuinely private search has on the rest of the market.
CHANGING THE GAME
Google has been spooked by the now rapidly-growing anti-surveillance backlash, which could snowball if it reaches critical mass, causing catastrophic problems for businesses that rely wholly on tracking.
The Big G is already pumping money into new advertising mechanisms that don't involve third party cookies, or the current theme of inscrutable corporate thugs (or “carefully selected partners”, to quote their official reference) stalking us around the Web. One of Google's alternative proposals - FLoC - has been criticised by the Electronic Frontier Foundation, and rightly so I believe. But this does show that if enough people reject aggressive surveillance, even the biggest powers have to start listening, and changing.
WHO'S AT RISK?
The companies who have most to lose if Brave gets this right, are not, however, Google or Microsoft. They're the current crop of “private search engines”. If Brave's results are not perceived by the average user as any worse than DuckDuckGo's, then surely, only publicity will save the tracker-reliant DDG from being dumped in favour of the independent Brave. And Brave is already a big enough brand to publicise a rival to DuckDuckGo.
Although DDG does now have a userbase who moved across due to Google's results filtering (especially in political contexts), its core user is still motivated by privacy. Once you introduce a new level of privacy in search - which is what Brave looks set to do - that user may well move on.
Google and Microsoft's core search user doesn't (yet) care about privacy, so the impact for these larger brands would be less immediate. Plus Google and Microsoft are diverse, and have a lot of other offerings beside search.
Ever-focused on raking in cash, however, Brave has already teased:
“We will offer Brave Search to power other search engines.”
That means if push came to shove, the likes of DuckDuckGo could switch from their current, tracker-supplied results, to Brave results. And I suspect Brave's long term plan is to subtly shame the current “private search engines” and a range of new ones, into doing exactly that. Whilst the public might benefit, I don't know whether DDG itself would prefer to be “Brave's bitch” than “Microsoft's bitch”, so to speak.
So Brave's move may instead prompt a healthier option than everyone using the same index. It might finally herald an era in which every “private search engine” generates its own results.
We need to see how well Brave's community-ranked index works in practice before we know how the current “private search engines” will be affected. One would not expect Brave's index to be anywhere near the size of Google's, but for the great majority of people it won't need to be. Most of the Google index is effectively inaccessible to most Google users, because simply, they have neither the inquisitiveness nor the advanced search knowledge to drill into it.
And we also need to think about how this could inconvenience Brave in ways that DDG, Startpage and Qwant are not inconvenienced. For example, try to serve a result removal request on most current "private search engines" and they'll say: "Nothing to do with us guv. We get our results from Microsoft/Google. Speak to them." But if Brave is using its own index, it will have to take responsibility for every result that's in violation of a law or regulation - including the right to be forgotten. That's going to involve human labour, which is expensive. Brave would have the option to avoid certain types of result - especially adult content, which is a quagmire of removal requests. Depending of the type of userbase Brave Search cultivates, blocking sensitive content could be an advantage. But it could also backfire.
DOES COMPETITION FROM BRAVE MATTER TO GOOGLE?
Yes. Google especially will want to hit back at Brave's projected shake-up of the search market, because on paper it threatens the leading search engine's power in a way no “private search engine” has before.
If, like me, you consider the current “private search engines” merely to be somewhat de-aggressed facades for Google or Bing, you'll conclude that they haven't changed the power dynamics at all. They don't threaten the big powers, because they depend on them for survival. DuckDuckGo won't criticise Microsoft. Startpage won't criticise Google. That's the control the big powers still have over their supposed “rivals”. Which are not really rivals at all. DDG, Startpage and Qwant are Microsoft and Google's bedfellows. Not their competition. And if they ever did become competition, their essential supplies would be cut off so fast that you wouldn't even see the scissors move.
For Google, the danger with Brave is that the Almighty G has no control over the situation. Google also stands to lose its place as default search on Brave Browser, and potentially on every other privacy-focused browser that wants to be taken seriously. Once that kind of thing starts, it could eat into the big brand's market share pretty quickly.
And the public might - just might - accept a broad Web-search system that is not subject to any of the tenets Google uses to control the digital landscape. That collection of tenets also allows Google to subordinate Bing, so it's a major source of digital domination. Even though Bing is technically independent from Google, it still exists in a world where Google dictates “SEO” practices. Bing is subservient to Google in having to observe and accommodate webmaster behaviour, as shaped by Google.
But if Brave Search ranks content based on mass human input rather than a mechanical interpretation of supposed qualitative indicators (i.e. backlinks), it may in time deliver better results than Google for the majority of users. That could erode Google's control not only of the search market, but also of the publishers themselves. Which could heavily disrupt the Bing/Yahoo system too. And most importantly, it could enormously benefit the public.
Many people who are not publishers remain unaware of how much control Google has over the way Web content is written and presented. Of how stifling Google's search engine optimisation requirements have been for creative innovation on websites.
GOOGLE VULNERABILITIES
Google has delivered many wonderful things for free, but it has also played God, imposing evermore strict guidelines on the way content should be structured and worded. As a publisher, you can either follow Google's guidelines, or you can not follow them, and be trampled out of view on the overwhelmingly dominant search engine, by publishers who do.
High end search optimisation is also now so technical and mathematical in its requirements that it can be considered anti-art, anti-comedy and anti-creative. Artists, humourists and creatives tend to think laterally rather than technically, and so are far less likely to understand an SEO concept like “structured data” than, say, a financial organisation would be.
Think about it. How many times have you ever Googled something and found a breathtakingly witty and clever title? Never? That's not because no one can think of witty, clever titles. It's because according to the Google system of search optimisation, the more mathematical a title's keyword makeup, the better the post performs in terms of visibility. So all titles are being written for machines to understand. Not for the entertainment of humans. That's why they're so phenomenally predictable.
How many times have you Googled a solution to a problem, and clicked on a result, only to have to read through twelve paragraphs of “why you would want to do this” before you get anywhere near the useful info? I mean, I'm guessing you already know why you want to solve your problem, right?… But again, it's not that the publishers don't want to do better. They know that twelve paragraphs of pointless waffle is annoying. But if they don't feed all that excruciating crap to the machines that assess content relevance, they don't get into the results, and you don't find their solution at all.
These compromises, along with countless others, are absolutely issues of Google lazily telling publishers to produce mechanical content rather than investing in a new system that recognises and gives full licence to flair. The top result on Google is not there because it's brilliant. It's there because it has the most backlinks. And backlinks can be obtained through all sorts of manipulative practices, including bribery, as Google knows very well. It's a sham.
But that also renders Google vulnerable to a system of mass human ranking, which doesn't need to place restrictions on how people write and present their work. It could, indeed, be the very vibrancy of searchable art, humour, creativity and fresh presentation that delivers a heavy blow to Google Search.
At present, we all turn to social media for that offbeat and innovative stimulation. But there's no reason why a search engine couldn't deliver it with better targeting and less wading through junk. Whether Brave's system will be the one to deliver that heavy blow I don't know. But Google has that vulnerability, and if people can find a more human, less mechanical world of enlightenment, the impact could be huge.
CHAPTER THREE
However this turns out for Brave, it will mark a new chapter in the story of Web search. And since many would argue that we've only ever had two chapters up until now (Pre-Google and Google), it promises to be a pretty dramatic development. Maybe not for Brave, and maybe not immediately, but it's a precedent. The first noticeable search brand saying: actually, maybe the era of “if you can't beat 'em, join 'em” is over. Perhaps the only thing you need in order to beat Google, is not to join them.” After all, Instagram snubbed Google, and look at it now.
People used to say that the letters in Bing stood for 'Because It's Not Google'. But Bing really is Google under another name, and partially under separate control. I look forward to the day when that monopoly of mechanical, outdated, 1990s search ranking breaks. And however small a step Brave's shift into Web search turns out to be, I get the sense that it will ultimately be very, very significant.