"scraping" feature

source: local ♦ tags: #meta #tech

#1 )) Name: Anonymous @ 2020-12-15 22:44

One interesting feature could be a scraper that pulls threads from other textboards, similar to Fuuka (warosu.org) or FoolFuuka.

Archivers serve a bit more of an obvious need on 4chan -- threads usually expire within a week, interesting comments can be removed by janitors and moderators, etc. But they can still fit into the general scope of multichan, which is to build a large database of textboard threads which can then be filtered by origin site or board (tags / topics).

simply duplicating these boards is nothing special:
- https://dis.4ct.org/prog/
- http://dis.tinychan.org/prog/
- https://4-ch.net/code/
- https://freesoftwarefoundation.org/prog
- https://textboard.org/prog/

.... BUT archiving them all and pooling them together WOULD be a fairly novel tool in itself, let alone as an extension of multichan.

#2 )) Name: Anonymous @ 2020-12-16 08:31

another programming textboard discovered:

#2,1 )) Name: Admin !cRwJk8JEBs @ 2020-12-19 10:11 🌎 0chan

Scraper can convert all Kareha boards to JSON. Next is emanon (post office, saovq, 4ct)...

#2,2 )) Name: Anonymous @ 2021-05-25 22:16 🌎 0chan

Any suggestions for non-Multich boards to scrape into the network?

This list could be a good place to look for URLs:
> http://xiongnu.org/overtext/

#3 )) Name: Anonymous @ 2021-05-31 12:01


> { host-board-thread[0]: {"url", "title", "replies", "created", "last_reply"}}
> { host-board-thread[x]: {"name", "email", "time", "message"}}

#4 )) Name: dev @ 2021-06-01 13:44

What would be good imageboard(s) to target?

#4,1 )) Name: Anonymous @ 2021-06-23 03:34 🌎 kuzlol



You need to solve the captcha before you can post.