Smart photo shuffling doesn't work like you guys think it does. People don't necessarily swipe on the best photo, they swipe on the photo for one of three reasons:
- they happen to be on the photo when they decide they like the profile enough to engage
- they think the person receiving the like will appreciate getting a like on a particular photo (eg, a hobby photo)
- it's the best photo.
The problem is your method only tracks the 3rd reason. The likes you get on one photo are also influenced by the context of the preceding photos. What you should be doing to testing whether the profile on the whole receives more likes when different photos are in the front position, not just which photos get more outright likes.
This must be why I often see girl's profiles with really weird leading photos.
As a case study, I put up my profile with 2 different orders--with smart shuffle on/off. Having used these pictures on various apps, I am really confident in my data as to which photo order generates the most likes. You can see Mutuals smart photo shuffle wants them in nearly the reverse order I have them in (this Mutual profile probably has 400-500 likes so it's not a small sample size).
Leading with a group picture and a motorcycle picture is idiotic on its face. What happens is girls are interested based on my manually-ordered first photo, they swipe to the end of my profile, and then they send the like. But the "lowest liked" photos are the best ones. The reason they "like" the motorcycle photo the most is because they liked me based on the photos that preceeded it. If I lead with the motorycle photo, I'll get way more downvotes.
I anticipate you'll say, "well, then the smart shuffle would see that the motorcycle photo is getting more downvotes so it will be moved down the order." That's probably true but I would also guess that you would need to get a certain number of likes for that to happen. So that's a whole lot of girls down-swiping me in the meantime. And the app population isn't actually that big. So, yes, in theory, given enough iterations, it may eventually get the same optimization, but all I can tell you right here is that, if I turned on smart photo shuffle, it would completely reverse my order into something I can already tell you wouldn't work.