I am several hundred opossums in a trench coat

  • 5 Posts
  • 74 Comments
Joined 1 year ago
cake
Cake day: July 1st, 2023

help-circle




  • Making a breaking change to the mobile API also breaks old outdated installations of the app. Websites and their APIs are usually synced, apps not so.

    If they were really motivated to stop your method, they could just obfuscate the frontend with webpack and break your scraper every time they make an update.


  • I suspect that any of the methods proposed here would be prone to a C&D, but IMO the safest legally would probably be the RSS method (not a lawyer though). Reddit’s RSS feeds are public, documented, and available without the need for private APIs, authentication, or an API key, so I don’t see how they could claim that a wrapper is unauthorised/illegal. Documenting their private API however seems like a gray area. Google LLC v. Oracle America, Inc. found that APIs are copyrightable, but this use may constitute fair use.


  • Emily@lemmy.blahaj.zonetoTechnology@lemmy.worldUnofficial Reddit API
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    2
    ·
    edit-2
    5 days ago

    Is there a reason you’re scraping data rather than attaching a network sniffer/reverse engineering the official apps and documenting the results? Or map the RSS feed to an API? The main thrust behind my comment is that I think scraping is pretty fragile, so I’m interested as to why other options are infeasible.


  • Thank you for adding this! If people want a real life example of the effect shown in this pseudocode, here is a side-by-side comparison of real production code I wrote and it’s decompiled counterpart:

        override fun process(event: MapStateEvent) {
            when(event) {
                is MapStateEvent.LassoButtonClicked -> {
                    action(
                        MapStateAction.LassoButtonSelected(false),
                        MapStateAction.Transition(BrowseMapState::class.java)
                    )
                }
                is MapStateEvent.SaveSearchClicked -> {
                    save(event.name)
                }
                // Propagated from the previous level
                is MapStateEvent.LassoCursorLifted -> {
                    load(event.line + event.line.first())
                }
                is MapStateEvent.ClusterClick -> {
                    when (val action = ClusterHelper.handleClick(event.cluster)) {
                        is ClusterHelper.Action.OpenBottomDialog ->
                            action(MapStateAction.OpenBottomDialog(action.items))
                        is ClusterHelper.Action.AnimateCamera ->
                            action(MapStateAction.AnimateCamera(action.animation))
                    }
                }
                is MapStateEvent.ClusterItemClick -> {
                    action(
                        MapStateAction.OpenItem(event.item.proposal)
                    )
                }
                else -> {}
            }
        }
    

    decompiled:

        public void c(@l j jVar) {
            L.p(jVar, D.f10724I0);
            if (jVar instanceof j.c) {
                f(new i.h(false), new i.r(c.class, (j) null, 2, (C2498w) null));
            } else if (jVar instanceof j.e) {
                m(((j.e) jVar).f8620a);
            } else if (jVar instanceof j.d) {
                List<LatLng> list = ((j.d) jVar).f8619a;
                j(I.A4(list, I.w2(list)));
            } else if (jVar instanceof j.a) {
                d.a a7 = d.f8573a.a(((j.a) jVar).f8616a);
                if (a7 instanceof d.a.b) {
                    f(new i.j(((d.a.b) a7).f8575a));
                } else if (a7 instanceof d.a.C0058a) {
                    f(new i.a(((d.a.C0058a) a7).f8574a));
                }
            } else if (jVar instanceof j.b) {
                f(new i.k(((j.b) jVar).f8617a.f11799a));
            }
        }
    

    keep in mind, this was buried in hundreds of unlabeled classes and functions. I was only able to find this in a short amount of time because I have the most intimate knowledge of the code possible, having written it myself.


  • It’s not impossible, just very labour intensive and difficult. Compiling an abstract, high level language into machine code is not a reversible process. Even though there are already automated tools to “decompile” machine code back to a high level language, there is still a huge amount of information loss as nearly everything that made the code readable in the first place was stripped away in compilation. Comments? Gone. Function names? Gone. Class names? Gone. Type information? Probably also gone.

    Working through the decompiled code to bring it back into something readable (and thus something that can be worked with) is not something a lone “very smart person” can do in any reasonable time. It takes likely a team of smart people months of work (if not years) to understand the entire structure, as well as every function and piece of logic in the entire program. Once they’ve done that, they can’t even use their work directly, since to publish reconstructed code is copyright infringement. Instead, they need to write extremely detailed documentation about every aspect of the program, to be handed to another, completely isolated person who will then write a new program based off the logic and APIs detailed in the documentation. Only at that point do they have a legally usable reverse engineered program that they can then distribute or modify as needed.

    Doing this kind of reverse engineering takes a huge amount of effort and motivation, something that an app for 350 total sneakers is unlikely to warrant. AI can’t do it either, because they are incapable of the kind of novel deductive reasoning required for the task. Also, the CarThing has actually always been “open-source”, and people have already experimented with flashing custom firmware. You haven’t heard about it because people quickly realised there was no point - the CarThing is too underpowered to do much beyond its original use.










  • Yeah see, I don’t think you get it. First of all the term has existed across multiple generations at this point, and really only unifies discussions of hegemonic masculinity that have spanned far longer.

    Secondly, and more importantly, toxic masculinity has nothing to do with the “basis of gender”, unless of course you’re claiming that these traits are inherent to males, in which case I suggest you start with “The Second Sex” and work your way up to a real conversation. To put it simply for you, toxic masculinity is just a term used to encompass certain behaviours, and (more importantly) how they are taught and reinforced. It’s obviously more complex than that, I haven’t even mentioned the study of how the rigid enforcement of these behaviours can negatively affect men, but I suggest you learn from a book instead of random women on Lemmy.






  • As a moderator of a couple communities, some basic/copypasta misbehaviour is caught by automated bots that I largely had to bootstrap or heavily modify myself. Near everything else has to be manually reviewed, which obviously isn’t particularly sustainable in the long term.

    Improving the situation is a complex issue, since these kinds of tools often require a level of secrecy incompatible with FOSS principles to work effectively. If you publicly publish your model/algorithm for detecting spam, spammers will simply craft their content to avoid it by testing against it. This problem extends to accessing third party tools, such as specialised tools Microsoft and Google provide for identifying and reporting CSAM content to authorities. They are generally unwilling to provision their service to small actors, IMO in an attempt to stop producers themselves testing and manipulating their content to subvert the tool.