So, in the era of increasingly good AI powered tools and general search engines full of SEO spam, last week I started creating something little old school and against the trends.

For now It’s a have-fun-and-find-out project that main aim is to provide good search results for general web development queries with a special focus on independent blog authors.

The thesis is that no SEO spam website is in the index, which will already filter out most annoying noise on Google/Bing.

Search results are grouped per type: docs, blogs and magazines (e.g. blog platforms or bigger websites).

For now it’s far from being done in terms of having a full index, but in most cases it already replaces my go-to search engine when I’m looking up some stuff during work.

I’m looking forward hearing out what y’all think and if you think it makes sense overall I can only encourage you to post some links to blogs or docs that are still missing in the index. I’m more than happy to add it to the crawler.

Responds like: “nei, total shit, who would need that” also accepted but constructive critique more appreciated ;)

EDIT: everyone many thanks for all your voices and comments. I’m super grateful for all of them and happy that we have such place like Lemmy!

  • Kissaki@feddit.de
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    11 months ago

    I have also a different problem, dev.to has a lot of good resources but also tons of SEO spam and low quality content. It’s also freaking huge and while it was for some time in the index I had to remove it and think about it some more.

    Yeah, a public platform is unlikely to provide consistent content. If curation is not an explicit goal and practice there, I would not include them for the reasons you mentioned.

    If indexing could happen not on domain but with more granular filters - URL base paths - that may be viable. Indexing specific authors on devto.

    • sznowicki@lemmy.worldOP
      link
      fedilink
      arrow-up
      2
      ·
      11 months ago

      Good idea. I had this thought once to do some narrow indexing of websites, e.g. stack overflow is a big issue, indexing all of this is crazy, picking up some specific tags on the other hand feels like tons of work. In the end I adjust the whole project as it grows with hope that after every tuning it gets better.

      As long as I have fun with it I’ll continue :D

      • Kissaki@feddit.de
        link
        fedilink
        English
        arrow-up
        2
        ·
        11 months ago

        Of course - cutting scope is a good call to keep it manageable and fun, and not end up with creep and what you wanted to evade in the first place. :)