"Success is not final, failure is not fatal: It is the courage to continue that counts." - Winston Churchill

• Not quite sure whether the above quote is real but you can buy a framed portrait of it on amazon just in case you felt you needed it 🤷

At the end of all this turmoil I can happily say that I have a trained model that CAN place shapes over the body segmentation alpha mask output of a body segmentation model AND can do so with about 90% accuracy in patterns that are not so displeasing.

While some might wonder whether this is a triumph worthy of praise I can certainly say that to the person who has trained this model it is (I trained the model just in case you were not sure)

Why was I even attempting to do this in the first place?

• The project first began fully on the fashion design end of the spectrum. I had been experimenting with creating a garment that was fully modular and made up of many individual pieces that can be attached together in many different ways. Sort of like a puzzle clothing concept.
• The main issue I was having with the concept was while 10 pieces of cloth could come together to make virtually any piece of clothing, actually putting the pieces together in a meaningful way proved much harder than it seemed

The early stages

• As I described in my previous post on the matter, the start was rougher than I expected. Not only was I attempting a task that was quite challenging to begin with, but I also had no idea how to even get started.

Tried and tried again | Notion

Automatic placement

• At a certain point I started to wonder more carefully at what I was feeding the model as an input and what I was trying to have it achieve. Through my training journey I had gotten tired and lazy. What was the point of placing 5 shapes per example by hand on a canvas only to have the model not train for some obscure issue deep in the p5.js library or have the model overfit instantly to one pattern only? I set aside some time and devised, with the help of chatGPT and claude.ai, an algorithm that could automatically place these shapes on the areas of the canvas that contained a human (aka alpha = 0). Dear reader before you start wondering whether this algorithm did not do exactly the thing I wanted the model to do I assure you that my vision was of automatic shape placement in a meaningful manner not just randomly on spots that contained low alpha.
• This automatic placement helped a lot and allowed me to train models with more than 100 example each! But alas I was still not getting any closer to my goal. The model would overfit instantaneously and the loss functions were comical at best (ranging from value of 7,00,000 to 0.01 in two epochs)

If training sample size is not the issue then what is?

• Why should my model be expected to generate an array of 10 shapes and place them all at once when it does not even know the positions of the other shapes it is generating coordinates for?
• I switched to a logic that focused on this idea that I got after thinking heavily about what my automatic placement script was doing. Unfortunately still not much better results wise but I felt I was getting closer.

You can always encode more data!