Greg Rutkowski, a digital artist known for his surreal style, opposes AI art but his name and style have been frequently used by AI art generators without his consent. In response, Stable Diffusion removed his work from their dataset in version 2.0. However, the community has now created a tool to emulate Rutkowski’s style against his wishes using a LoRA model. While some argue this is unethical, others justify it since Rutkowski’s art has already been widely used in Stable Diffusion 1.5. The debate highlights the blurry line between innovation and infringement in the emerging field of AI art.
I don’t fully understand how this works, but if they’ve created a way to replicate his style that doesn’t involve using his art in the model, how is it problematic? I understand not wanting models to be trained using his art, but he doesn’t have exclusive rights to the art style, and if someone else can replicate it, what’s the problem?
This is an honest question, I don’t know enough about this topic to make a case for either side.
TL;DR The new method still requires his art.
LoRA is a way to add additional layers to a neural network that effectively allow you to fine tune it’s behaviour. Think of it like a “plugin” or a “mod”
LoRas require examples of the thing you are targeting. Lots of people in the SD community build them for particular celebrities or art styles by collecting examples of the that celebrity or whatever from online.
So in this case Greg has asked Stable to remove his artwork which they have done but some third party has created an unofficial LoRA that does use his artwork to mod the functionality back in.
In the traditional world the rights holder would presumably DMCA the plugin but the lines are much blurrier with LoRA models.
Great explanation, thanks!
You’re pretty spot on. It’s not much different from a human artist trying to copy his style by hand but without reproducing the actual drawings.
Do you know how they recreated his style? I couldn’t find such information or frankly have enough understanding to know how.
But if they either use his works directly or works created by another GAI with his name/style in the prompt, my personal feeling is that would still be unethical, especially if they charge money to generate his style of art without compensating him.
Plus, I find that the opt-out mentality really creepy and disrespectful
I still have trouble understanding the distinction between “a human consuming different artists, and replicating the style” vs “software consuming different artists, and replicating the style”.
That’s really the big thing, not just here but any material that’s been used to train on without permission or compensation. The difference is that most of it is so subtle it can’t be picked out, but an artist style is obviously a huge parameter since his name was being used to call out those particular training aspects during generations. It’s a bit hypocritical to say you aren’t stealing someone’s work when you stick his actual name in the prompt. It doesn’t really matter how many levels the art style has been laundered, it still originated from him.
It is unconditionally impossible to own an artistic style. “Stealing a style” cannot be done.
Just wait until you can copywrite a style. Guess who will end up opening all the styles.
Spoiler, it’s wealthy companies like Disney and Warner. Oh you used cross hatching? Disney owns the style now you theif.
Copyright is fucked. Has been since before the Mickey mouse protection act. Our economic system is fucked. People would rather fight each other and new tools instead of rallying against the actual problem, and it’s getting to me.
Generally speaking, the way training works is this:
You put together a folder of pictures, all the same size. It would’ve been 1024x1024 in this case. Other models have used 768z768 or 512x512. For every picture, you also have a text file with a description.
The training software takes a picture, slices it into squares, generates a square the same size of random noise, then trains on how to change that noise into that square. It associates that training with tokens from the description that went with that picture. And it keeps doing this.
Then later, when someone types a prompt into the software, it tokenizes it, generates more random noise, and uses the denoising methods associated with the tokens you typed in. The pictures in the folder aren’t actually kept by it anywhere.
From the side of the person doing the training, it’s just put together the pictures and descriptions, set some settings, and let the training software do its work, though.
(No money involved in this one. One person trained it and plopped it on a website where people can download loras for free…)
I don’t, but another poster noted that it involves using his art to create the LoRA.
I don’t know about creepy and disrespectful, but it does feel like they’re saying “I know the artist doesn’t want me to do this, but if he doesn’t specifically ask me personally to stop, I’m going to do it anyway.”