Skip to content

fix: PhotoMaker: avoid GGML_ASSERT if trigger word was not found in prompt - #1835

Merged
leejet merged 1 commit into
leejet:masterfrom
akleine:master
Jul 30, 2026
Merged

fix: PhotoMaker: avoid GGML_ASSERT if trigger word was not found in prompt#1835
leejet merged 1 commit into
leejet:masterfrom
akleine:master

Conversation

@akleine

@akleine akleine commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Currently, a missing trigger word ("img") in PhotoMaker's prompt is not handled correctly (see below).
I added an additional check for class_idx >= 0 when creating the class_token_mask.

That leads to this message:
[WARN ] photomaker_extension.cpp:232 - Turn off PhotoMaker for this request
and continuation of the program just like in similar situations (e.g. "Photomaker ID Stacking failed" etc.) .

Related Issue / Discussion

Note: class_idx stays at initial values of -1 if we do not get the trigger word in the loop before.

Additional Information

Before this little patch:

[DEBUG] bpe_tokenizer.cpp:208  - split prompt "img" to tokens ["img</w>", ]                                                                                                                   
[DEBUG] conditioner.hpp:347  - parse 'man is playing a guitar' to [['man is playing a guitar', 1], ]                                                                                          
[DEBUG] bpe_tokenizer.cpp:208  - split prompt "man is playing a guitar" to tokens ["man</w>", "is</w>", "playing</w>", "a</w>", "guitar</w>", ]                                               
/home/usr/stable-diffusion.cpp/src/extensions/photomaker_extension.cpp:99: GGML_ASSERT(it != tokens.end()) failed                                                                             
. . .                                                                                                                                                                                           
#0  0x0000713df0910813 in __GI___wait4 (pid=4095, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30                                                                
#1  0x0000556b6c5dace3 in ggml_print_backtrace ()                                                                                                                                             
#2  0x0000556b6c5dae87 in ggml_abort ()                                                                                                                                                       
#3  0x0000556b6b87d00c in PhotoMakerExtension::prepare_condition(GenerationExtensionConditionContext&) ()                                                                                     
#4  0x0000556b6b6117cf in prepare_image_generation_embeds(sd_ctx_t*, sd_img_gen_params_t const*, GenerationRequest*, SamplePlan*, ImageGenerationLatents*, RefImageParams const&) ()          
#5  0x0000556b6b61a706 in generate_image ()                                                                                                                                                   
#6  0x0000556b6b404130 in main ()                                                                                                                                                             
[Inferior 1 (process 3985) detached]                                                                                                                                                          
./0_NEWTON_MAN_FRU: Zeile 11:  3985 Abort (core dumped) ./sd-cli -m ~/SD_models/sdxl/zavychromaxl_v100.safetensors --vae ~/SD_models/sdxl/sdxl_vae.safetensors --clip-on-cpu --steps 20 -v --s

Now we have:

[DEBUG] bpe_tokenizer.cpp:208  - split prompt "img" to tokens ["img</w>", ]                                                                                                                   
[DEBUG] conditioner.hpp:347  - parse 'man is playing a guitar' to [['man is playing a guitar', 1], ]                                                                                          
[DEBUG] bpe_tokenizer.cpp:208  - split prompt "man is playing a guitar" to tokens ["man</w>", "is</w>", "playing</w>", "a</w>", "guitar</w>", ]                                               
....                                                                                                                                                                                          
[WARN ] photomaker_extension.cpp:231  - PhotoMaker trigger word 'img' was not found in prompt                                                                                                 
[WARN ] photomaker_extension.cpp:232  - Turn off PhotoMaker for this request                                                                                                                  
[DEBUG] conditioner.hpp:347  - parse 'man is playing a guitar' to [['man is playing a guitar', 1], ]                                                                                          
[DEBUG] bpe_tokenizer.cpp:208  - split prompt "man is playing a guitar" to tokens ["man</w>", "is</w>", "playing</w>", "a</w>", "guitar</w>", ]                                               
[DEBUG] ggml_extend.hpp:2175 - clip compute buffer size: 1.42 MB(RAM)                                                                                                                         . . .

Checklist

…d in prompt

Before this little patch:
[DEBUG] bpe_tokenizer.cpp:208  - split prompt "man is playing a guitar" to tokens ["man</w>", "is</w>", "playing</w>", "a</w>", "guitar</w>", ]
/home/usr/stable-diffusion.cpp/src/extensions/photomaker_extension.cpp:99: GGML_ASSERT(it != tokens.end()) failed
...
[Inferior 1 (process 3985) detached]
./0_NEWTON_MAN_FRU: Zeile 11:  3985 Abort (core dumped) ./sd-cli -m ~/SD_models/sdxl/zavychromaxl_v100.safetensors --vae ~/SD_models/sdxl/sdxl_vae.safetensors --clip-on-cpu --steps 20 -v --seed 44 --cfg-scale 5 --photo-maker ~/SD_models/loras_sdxl/photomaker-v1.safetensors --pm-id-images-dir ./assets --pm-style-strength 5 -p "man is playing a guitar"

Now:
[WARN ] photomaker_extension.cpp:231  - PhotoMaker trigger word 'img' was not found in prompt
[WARN ] photomaker_extension.cpp:232  - Turn off PhotoMaker for this request
...and continue here and write output file
@leejet
leejet merged commit 735a4ef into leejet:master Jul 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants