Drew DeVault is writing a new search engine

whats wrong with sql, postgres works fine on scale for my purposes and works fine for matrix/pleroma as well

>My search engine of choice? A private SearX (pronounced sirks, btw) instance that scrapes Mojeek and Wiby, of course.

Attached: black hole.jpg (1200x1219, 226.01K)

No, you are what's wrong with the internet. Your dream is over-balkanization of the internet.

you clearly never had a pleroma with over 100GB of postgres data. the scaling properties of postgres is not even fucking close to good enough for something like text search for web content. yes even with rum indices.

He doesnt have a real job and just freeloads and hopes that simps will pay money for his trash vapourware. Not hard to start a bunch of awful projects that no one will use or give a shit about

what do you recommend? i'm working on something that needs to scale and i'd appreciate the advice

Who???

and your dream is literally cyberpunk 2077 and ill take overbalkanization over it

if you can figure out how to give sql syntax to something akin to a dht, that would be so much more useful, that way you can fan out the load horizontally... forever.

NO, my dream is a service for every need. You want to find an answer to a technical question? You go to Google. You want to see kids being abused or political bullshit? You go to Yandex. You want to see ....idk.... you use Bing.

Your dream of overbalkanization would lead to a situation of the likes of the proto-internet where no service offers a decent solution to any problem.

is there a working implementation of this or are you just theorycrafting?

If youre going to go full terry davis make a web browser from scratch.

your dream of specialized services only leads to monopolization. i don't agree with your vision of "overbalkanization", in my eyes competition only leads to improvements

apache spark but like...
>apache and hadoop trash piles
i wish i could allocate time to make a lookalike in golang

Just use SearX to consolidate all search engines into a unified interface and pick and choose from the assortment as you see fit.

>How the fuck he has time for so many projects???
Most of his projects are fairly small in scale, he's self-employed so he probably spends an hour a day on his main job (sourcehut) and the rest of his day is spent on programming other stuff

You know that his search engine ill be a failure, right? If you're a normie you use Google, if you're a Pedophile you use Yandex, if you're an altright you use Yandex, if you're an idiot you use Bing.

This is a winning formula.

I dont see how anyone can comoete with google when they handle 100 oetabytes of data each day for their search engine.
Its too late for that.

google is shit nowadays, i think they must have changed their algorithm a couple months ago because now i find searching for technical questions so much harder than what it used to be, also it now ignores quotation marks and even if it doesn't find anything relevant it still returns absolutely useless results.
it's like they are actively trying to waste my time and annoy me... i was actually considering to make a programming-focused search engine to cope with that, but if i don't have to bother building it that would be nice

Doesn't matter, once Apple's comes out there will be zero desire for any other search engine
It's over nerds