Greg Rutkowski, a digital artist known for his surreal style, opposes AI art but his name and style have been frequently used by AI art generators without his consent. In response, Stable Diffusion removed his work from their dataset in version 2.0. However, the community has now created a tool to emulate Rutkowski’s style against his wishes using a LoRA model. While some argue this is unethical, others justify it since Rutkowski’s art has already been widely used in Stable Diffusion 1.5. The debate highlights the blurry line between innovation and infringement in the emerging field of AI art.
What blurry line? An artist doesn’t what his art stolen from him. Seems pretty cut and dry to me.
I don’t disagree but stolen is a bit of a stretch
I don’t fully understand how this works, but if they’ve created a way to replicate his style that doesn’t involve using his art in the model, how is it problematic? I understand not wanting models to be trained using his art, but he doesn’t have exclusive rights to the art style, and if someone else can replicate it, what’s the problem?
This is an honest question, I don’t know enough about this topic to make a case for either side.
TL;DR The new method still requires his art.
LoRA is a way to add additional layers to a neural network that effectively allow you to fine tune it’s behaviour. Think of it like a “plugin” or a “mod”
LoRas require examples of the thing you are targeting. Lots of people in the SD community build them for particular celebrities or art styles by collecting examples of the that celebrity or whatever from online.
So in this case Greg has asked Stable to remove his artwork which they have done but some third party has created an unofficial LoRA that does use his artwork to mod the functionality back in.
In the traditional world the rights holder would presumably DMCA the plugin but the lines are much blurrier with LoRA models.
Great explanation, thanks!
You’re pretty spot on. It’s not much different from a human artist trying to copy his style by hand but without reproducing the actual drawings.
Do you know how they recreated his style? I couldn’t find such information or frankly have enough understanding to know how.
But if they either use his works directly or works created by another GAI with his name/style in the prompt, my personal feeling is that would still be unethical, especially if they charge money to generate his style of art without compensating him.
Plus, I find that the opt-out mentality really creepy and disrespectful
I still have trouble understanding the distinction between “a human consuming different artists, and replicating the style” vs “software consuming different artists, and replicating the style”.
That’s really the big thing, not just here but any material that’s been used to train on without permission or compensation. The difference is that most of it is so subtle it can’t be picked out, but an artist style is obviously a huge parameter since his name was being used to call out those particular training aspects during generations. It’s a bit hypocritical to say you aren’t stealing someone’s work when you stick his actual name in the prompt. It doesn’t really matter how many levels the art style has been laundered, it still originated from him.
It is unconditionally impossible to own an artistic style. “Stealing a style” cannot be done.
Just wait until you can copywrite a style. Guess who will end up opening all the styles.
Spoiler, it’s wealthy companies like Disney and Warner. Oh you used cross hatching? Disney owns the style now you theif.
Copyright is fucked. Has been since before the Mickey mouse protection act. Our economic system is fucked. People would rather fight each other and new tools instead of rallying against the actual problem, and it’s getting to me.
Generally speaking, the way training works is this:
You put together a folder of pictures, all the same size. It would’ve been 1024x1024 in this case. Other models have used 768z768 or 512x512. For every picture, you also have a text file with a description.
The training software takes a picture, slices it into squares, generates a square the same size of random noise, then trains on how to change that noise into that square. It associates that training with tokens from the description that went with that picture. And it keeps doing this.
Then later, when someone types a prompt into the software, it tokenizes it, generates more random noise, and uses the denoising methods associated with the tokens you typed in. The pictures in the folder aren’t actually kept by it anywhere.
From the side of the person doing the training, it’s just put together the pictures and descriptions, set some settings, and let the training software do its work, though.
(No money involved in this one. One person trained it and plopped it on a website where people can download loras for free…)
I don’t, but another poster noted that it involves using his art to create the LoRA.
I don’t know about creepy and disrespectful, but it does feel like they’re saying “I know the artist doesn’t want me to do this, but if he doesn’t specifically ask me personally to stop, I’m going to do it anyway.”
His art was not “stolen.”
Nothing was stolen.
Drawing inspiration from someone else by looking at their work has been around for centuries.
Imagine if the Renaissance couldn’t happen because artists didn’t want their style stolen.
If I look at someone’s paintings, then paint something in a similar style did I steal their work? Or did I take inspiration from it?
No, you used it to inform your style.
You didn’t drop his art on to a screenprinter, smash someone else’s art on top, then try to sell t-shirts.
Trying to compare any of this to how one, individual, human learns is such a wildly inaccurate way to justify stealing a someone’s else’s work product.
If it works correctly it’s not a screenprinter, it’s something unique as the output.
The fact that folks can identify the source of various parts of the output, and that intact watermarks have shown up, shows that it doesn’t work like you think it does.
Does that mean the AI is not smart enough to remove watermarks, or that it’s so smart it can reproduce them?
LLMs and directly related technologies are not AI and possess no intelligence or capability to comprehend, despite the hype. So, they are absolutely the former, though it’s rather like a bandwagon sort of thing (x number of reference images had a watermark, so that’s what the generated image should have).
That’s debatable. LLMs have shown emergent behaviors aside from what was trained, and they seem to be capable of comprehending relationships between all sorts of tokens, including multi-modal ones.
Anyway, Stable diffusion is not an LLM, it’s more of a “neural network hallucination machine” with some cool hallucinations, that sometimes happen to be really close to some or parts of the input data. It still needs to be “smart” enough to decompose the original data into enough and the right patterns, that it can reconstruct part of the original from the patterns alone.
Thanks for the clarification!
LLMs have indeed shown interesting behaviors but, from my experience with the technology and how it works, I would say that any claims of intelligence being possessed by a system that is only an LLM would be suspect and require extraordinary evidence to prove that it is not mistaken anthropomorphizing.