Choosing Tokens Wisely: Good vs. Bad in Stable Diffusion
Written on
Chapter 1: The Impact of Token Choice
Your selection of a token when utilizing a Stable Diffusion model through Dreambooth plays a crucial role in both the training process and the final appearance of your subject.
It's essential to grasp how different tokens can influence outcomes, especially when training models.
Section 1.1: The Dangers of Bad Tokens
A poor choice of token, or an ineffective token, is one that has established connections to specific concepts within the model, such as objects, individuals, or events.
For instance, consider using a combination of a first and last name, like "anitabath" or "carrieoakey." While this can be effective, imagine if you share your name with a famous person, say Tom Cruise. Using the token "tomcruise" could lead to a strong pre-existing association in the model. Although with sufficient training, you might overwrite this association, initially, your results may resemble a bizarre mix of your subject's features and those of Tom Cruise.
Section 1.2: The Benefits of Good Tokens
In contrast, a good token is one that lacks strong associations and can yield seemingly random images. You may never find a token that generates absolutely nothing, but aiming for minimal associations is key.
Important Avoidances:
- Single letters, particularly common ones like "a," which frequently appear in prompts (e.g., "<a> photo of <a> man sitting on <a> chair in <a> cafe.").
- Common nouns and names.
- Well-known individuals, both real and fictional.
- Abbreviations related to recognized brands or companies.
- Long, complex words that are not in the model's dictionary, as these may break into smaller tokens with strong associations.
According to the Dreambooth paper, commonly used English words (like "unique" or "special") are often suboptimal since the model must learn to separate them from their original meanings.
A risky strategy is to concatenate random English characters to create a unique identifier (e.g., "xxy5syt00"). However, the tokenizer might break these down into individual letters, leading to strong prior associations.
Let's extend that token example: "xxy5sytcar01230." The breakdown reveals that "car" is the most influential token.
Examples of Token Performance
Exploring a few token examples can illustrate their effectiveness:
- sks
- Bad Token: Strongly associated with rifles, particularly when paired with "woman."
- bnha
- Bad Token: Clearly linked to anime aesthetics across all variations.
- http
- Good Token: Generates random images, showing diversity across all classes.
- hta
- Good Token: Also produces random images without a clear association.
- hmv
- Bad Token: Strongly connected to a music store, affecting the results.
- httr
- Bad Token: Strong ties to an American football team, impacting output.
- rune
- Bad Token: A strong association with runes and fantasy imagery.
- inem
- Good-Okay Token: Yields random images, although with a slight focus on black and white.
- mib
- Bad Token: Strongly connected to "Men in Black."
- oue
- Good Token: Slightly emphasizes fashion when associated with women.
Ultimately, the token you decide to use is entirely your choice. If it proves effective, then it achieves its purpose. For those named Tom Cruise who are indifferent to overshadowing the more famous individual, feel free to proceed.
The example images presented here were generated using a Google Colab Notebook, which allows for token input and produces a grid of images based on that token. This model is based on Stable Diffusion 1.5.
Links
- sd-1–5-token-images.zip (a collection of over 600 images generated from various tokens)
- Stable Diffusion Dreambooth Token Checker
- Dreambooth Paper
For any inquiries or to learn more, consider joining the
Stable Diffusion Training Discord Server, where many knowledgeable individuals are ready to assist.
Chapter 2: Analyzing Token Design in Web3 Gaming
Understanding token design and its compliance implications is essential for the growth of web3 gaming.
The first video titled "Bad Token Design and Compliance Issues in Web3 Gaming" explores the challenges faced in token implementation and their effects on gaming environments.
Chapter 3: Revisiting Ineffective Token Strategies
Sometimes, recycling previously ineffective token strategies can lead to better outcomes.
The second video "MAGMML3 #12 - Recycling An Already Bad Token Trick" delves into how existing mistakes can be re-evaluated for future success.