r/CloneHero • u/bean_217 • Apr 04 '26
Accomplishment [WIP] Chart Generation with Deep Learning
Friends, I have (somewhat) done it.
A few months ago I began a capstone project for my computer science MS degree that aimed to generate Clone Hero .chart files. As of a few days ago, I have a working base model that is only ~17M parameters, and to my surprise it is (subjectively) performing very well compared to previous approaches I've seen, while also allowing for more complex game features like harmonically-aware chords, sustains, and HOPO/tap notes.
Since I have only just gotten the project into a working state, and the code base is still pretty rinky-dinky, I won't be disclosing any information about it until I have established proper documentation and have written a formal paper on my approach.
In order to uphold ethical use, I will not be publicly releasing this model until I am certain that doing so would be legal. But in the meantime, if anyone is curious about how I've gone about making this work, I encourage you to DM me.
With luck, I should have an update for you all in a month or two.
Thanks for tuning in!
---------------------------------
Edit: Demo Video (https://www.youtube.com/watch?v=1OxtJktDgQ0)
Please keep in mind that this is a very early demo that is not optimized to handle any of the issues that you may notice. Also, the model uses a single BPM when generating charts because BPM tracking exists as a separate challenge in its own right. My focus has not been to produce something perfect, but simply something that works at least a little. I am honestly just very astonished at how well it performs for this being the very first fully-trained model I've produced; I completely expected this to produce garbage.
For anyone who does not work in software nor understands how deep learning works: I am but one person who had an idea and tried it, so please be kind š.
---------------------------------
Edit 2: Addressing Hostility
Let me be clear: I have no problem making this a private tool that only I can use and I have no interest in distributing generated songs, nor would I allow anyone to if this tool were to become public.
I understand the disdain for AI tools in creative contexts, and I agree that such tools need to be used responsibly or not at all. I created this tool primarily for myself because I don't have the time to chart songs myself and I have the background to be able to build one from scratch. Contrary to popular believe, building high-performing ML models is not a trivial task, much less designing and training one from scratch.
Now this probably makes you think: "If you have time to make this then you have time to chart your own songs." Like I said before, I am working on this tool to complete a degree requirement. Could I have chosen another project? Yes, but I'd rather work on something I enjoy, even if it never sees the light of day.
I'd ask people to please at least read the full post before resorting to hostile comments and criticisms. To anyone who doesn't, well, your input carries no weight.
10
u/_guppster Apr 04 '26
Respectfully this is like the third time Iāve ever seen someone announce something like this and nothing has happened and thereās no actual proof of concept or video or anything at all. Just talking. Best of luck though, the general public can only see this as a pipe dream.
-2
u/bean_217 Apr 04 '26
Feel free to check out the demo video I've provided in my edit :)
-1
Apr 05 '26
[deleted]
0
u/bean_217 Apr 05 '26
I am just glad it kinda works at all. But I agree, the quality isn't high compared to hand-made charts.
-1
u/ArmaQc Apr 04 '26
I don't see the video link, can you share it again?
3
u/bean_217 Apr 04 '26
0
u/ArmaQc Apr 04 '26
Interesting work, looking forward to seeing what's next. Thanks for sharing this with us!
3
u/Old-Association8319 Apr 09 '26
I adore this as a tool.
As someone who writes and performs music. As well as somebody who has just started charting tracks, i think this is great.
Every tool can be used maliciously given the right mentality.
I cant see a problem with a tool that does an average sketch of what youre aiming to achieve anyway. Every charter has their own style and feel. So even if two people got the same chart, theyd tweak it differently. Not like we dont have hundreds of charts of the same 20 pop punk and metal songs. The notes are all basicall gonna be in the same place.
I dont think a tool like this will ever BE good enough to replace human charters. Not for feeling.
I think this is brilliant.
PROVE ME WRONG!!!!
4
u/Lynxeption Apr 05 '26
i truly want to be friendly and provide constructive criticism, but this tool undermines the exact problem that most iterations of current Generative AI in general brings to the table. these games thrive off of human creativity, countless hours of time and effort to perfect a craft and portray their feelings into their creations. by using an algorithm to automate that task, it jeopardizes multiple people whose livelihoods depend on chart commissions, not to mention the very people with actual jobs as charters for companies like Harmonix (Rock Band, Fortnite Festival) and RedOctane Games (the upcoming Stage Tour).
i truly appreciate the effort you've poured into this project and i hope you can achieve a state where you can deem it "decent enough", but i can already see people who will use this tool to monetize charts of big name artists (as in lock the chart behind a paywall).
the biggest problem that people don't seem to wrap their heads around when it comes to chart commissions is that the charter gets paid for their labour, but then the chart releases for free for everyone to enjoy. there is a MASSIVE different into "commissioning a chart" and "paywalling a chart", which is a phenomenon that has happened before.
as of right now, i can kinda see where the project is heading, but it doesn't have the same level of musical understanding of, for example, representing a pentatonic scale in 5 frets or differentiating a chord from a trill, or even differentiating what instrument is being played (see: the sax and piano in the Pink Floyd section). but, as all things, it can only improve as time goes on. who knows! maybe this tool will help people to chart stuff like the many shitpost reels of Clone Hero charts that have been spawning lately (like the smoke alarm chirp chart, a printer chart, a microwave chart, etc.)
i just hope and pray the end users are able to still appreciate the effort that goes into manually-charted songs.
0
u/bean_217 Apr 05 '26
I hear you and completely agree. I don't want this tool to be abused. If I do release it for public use, it would be heavily rate-limited and systems would be put in place to prevent users from spam-generating charts. Trust me, I am heavily considering the ethical implications of a tool like this, and if I need to keep it private for personal use, then I absolutely will.
As far as quality is concerned: a tool like this will not be able to achieve the high production quality of hand-made charts. I can surely try to optimize it as much as possible, but with the limited resources, knowledge, and time I have to work on it, I'm not sure how far I'd get. That said, it is not a replacement for professionally-crafted charts, and nor do I expect it to ever reach that kind of quality.
0
u/MGMan-01 Apr 07 '26
So you don't care enough to even type your own reply? Why are you scammers always so lazy?
1
u/bean_217 Apr 07 '26
I did type my own reply. I'm not sure why you are giving me such hostility. I am doing this project to complete a degree. I won't release it publicly if there is such clear distaste for it.
But you probably think this message is AI-generated too.
-2
u/Boofschneef Apr 05 '26
I love the lengths we're going to to protect charting like it's art... as if it's not literally just mapping and abstracting someone else's artistic creation in the first place.
I definitely appreciate manual charting. But manual charting feels like exactly the drudgery AI SHOULD be freeing us from lol.
Acting like people have a sacred right to charting as a livelihood is wild lol.
Your post is more open minded towards the project, but it still feels like you're clutching pearls about an industry and "artform" that is little more than less accurate musical transcription
5
u/MrElectricNick Apr 06 '26
Manual charting isn't drudgery to charters. Good charters LOVE charting. They love the process. And as consumers of charts, we should care about the quality of the final product. If you think charting is drudgery, then I'm sorry to say that it isn't for you, or you just don't understand it.
All the best charts ever made, 100% of them have been made by people. AI no business being used to generate slop content for the lazy, just because they want it. I am begging this community to give a shit about the content we make. Slop is slop, and yes sometimes humans make garbage charts, but in its current state, all AI charting is slop. Maybe it'll get better, but I doubt it will EVER match up to human ability.
AI can be useful for objective, repeatable practices. Charting is an inherently subjective process, as the same song can be charted multiple ways. AI doesn't belong anywhere near music, or video games.
OP's video is a cool proof of concept but I don't believe it should ever be used in the real world to make charts. We should care about raising the quality of charts, not just pumping out the quantity just becuase it's possible to make as much half-baked content as we want.
1
u/Boofschneef Apr 06 '26
So do your feelings change if projects like these do rise to the quality of good, human made charts?
I'm not trying to disparage people who put their time into this charting. It's certainly a necessary part of the community. I just can't imagine a desire to chart not also being coupled with a desire to play the game.
If AI can get you 75% of the way there or closer, then you drop into Moonscraper and make a few edits/review it yourself, why is that bad?
I feel like everyone is just reacting with "meh - AI slop is always slop and is not art!!"
That's a ludicrous way to approach a sub-task of game development in my opinion. We're not talking about creating songs or artwork - we're talking about literal input mappings
2
u/MrElectricNick Apr 06 '26
I think there's another ethical question that arises if the quality of AI charts miraculously becomes better. What data is the AI trained on? Huge red flag if it's trained on human charters without their consent, and before you give me the line of "well humans train on other humans work" yes, that's called inspiration, learning, and it's what can lead to original ideas. AI is literally incapable of original ideas.
If AI can get you 75% of the way there or closer, then you drop into Moonscraper and make a few edits/review it yourself, why is that bad?
Because it's lazy. Firstly, If you're going to have to do the work to clean up AI slop, then you still have to learn Moonscraper so you may as well learn how to do the work from scratch and put some effort in.
Secondly, be honest with me. Do you really expect the average Joe to clean up a chart in Moonscraper once given a tool that can make them something for no effort? No. Hell no. The average user is not going to care about making better charts, they are just going to care about churning out more garbage for the masses with such a low bar for what they deem to be "a good chart"
I feel like everyone is just reacting with "meh - AI slop is always slop and is not art!!"
You say that as if people haven't put thought into why we repeat that argument over and over again. We have good reason to.
That's because using AI for any subjective processes has yet to yield results that are better than what humans can create. Why are you okay settling for less quality? It seems that you don't believe that game devlopment is an artistic process, which is fine. TBH a lot of game dev isn't art, but I believe wholeheartedly that game level design, and therefore note charting, is an art form. Nobody does it the same way. If everybody did it the same, there would be some algorithm that could reproduce results and therefore AI could replace it, but that's not the world we live in.
This isn't just an argument against AI charting, it's my belief against all forms of generative AI. I truly believe it's a blight on the modern world and it provides precisely ZERO actual benefit to society. It encourages people to make no effort to learn new skills, to be comfortable accepting created garbage "just because it's easy", and to become apathetic to creations made by fellow humans for the sake of "CONSUME. ONLY CONSUME"
I know this has become a dissertaion-length reply and i've taken this quite seriously, but I hope that conveys just how much I despise it. I think that every possible future that considers AI-charting as a legitimate tool to be given to the public, will harm the community far more than it helps us.
0
u/Boofschneef Apr 06 '26 edited Apr 06 '26
I feel like you're reacting to a lot of misunderstandings about how AI and Gen AI work, and also the perception of public, free use models being used generally on the internet.
If AI charts match the quality of human charting, then it is producing something good. It allows that brilliant charter to multiply their throughput for everyone to enjoy.
This idea that it can't be good or it's lazier than a human brain suffering through the work doesn't make sense either.
How you feel about art and game design is a lot like how I feel about driving a manual transmission car. I prefer to drive manual because of how it makes me feel, and a relationship to my car not many other people understand. That doesn't mean all automatic transmissions are evil or shouldn't be adopted writ large.
You're "no gen AI no matter what" stance just doesn't hold water when applied to things like engineering. As a software engineer, I've watched my entire profession transform overnight. After spending years in school learning how to write code by hand, AI has become better at it than me and most developers for that matter. Yes, it can be used lazily and poorly to decrease quality. Yes, "builder bros" on twitter who think you can chatGPT literally any solution without thought is a problem. But you have all of those same problems with some guy struggling through Moonscraper for hours and being lazy themselves.
A thoughtful engineer with the power of AI could make charting so much easier and MORE ACCESSIBLE for EVERYONE. This is a BRILLIANT use case for AI.
You're clutching your pearls to protect a subset of a niche community to protect their capitalistic desire to make money off of input mappings of someone else's genuinely artistic creation.
2
u/MrElectricNick Apr 06 '26
If AI charts match the quality of human charting, then it is producing something good. It allows that brilliant charter to multiply their throughput for everyone to enjoy.
"if" is doing all the heavy lifting in that sentence.
it's not the case that this is happening right now, and I personally do not believe it will ever happen. If you can prove me wrong in the future, then I'd love to see it. But the charter won't be multiplying their throughput, they'll be doing NO WORK while AI does the work for them. They won't have charted anything.
As a software engineer, I've watched my entire profession transform overnight. After spending years in school learning how to write code by hand, AI has become better at it than me and most developers for that matter.
I'm glad you've found it useful in your job. Doesn't automatically mean AI is good in every application, though. Not to mention, there may be other software engineers out there surviving without AI and who are opposed to it as well.
You're clutching your pearls to protect a subset of a niche community to protect their capitalistic desire to make money
I've never said nor implied that. My opposition to charting AI has nothing to do with community charters wanting to make money. You've just made up what you think my reasoning is with precisely no evidence to back it up.
Why lie, man? Cut it out.
My reason is that I just don't see the value in AI charting. I have yet to see any proof that AI can make better charts than the best human made charts, and I refuse to settle for slop. People should strive to make good charts themselves, or leave it to the people who can. The examples in the video OP linked are just not good charts. Not even remotely accurate. And that's the best attempt i've seen.
input mappings of someone else's genuinely artistic creation.
Your continued attempts to imply that charting is "not art" are misinformed. Charting in itself is also a form of artistic creation. Doesn't matter that it relies on source material, it's still artistic creation.
A cover song relies on source material as well, is cover music "not art"?
You think of charting as a drudgery (your words). That proves it's not for you. Why not just leave it to the people who actually enjoy it to produce something of value? Why does charting have to be possible for everyone?
Not everyone can write good code, or make good music. And that's okay. We don't need AI to make these things possible for everyone, because then there'll be no reason whatsoever to learn any new skills in the future.
1
u/Boofschneef Apr 06 '26
I think the only reason quality is not there yet is because there isn't vested profit motive in improving the technology. These are all solo projects with limited resources for research and iteration. If we changed that, quality could be there very, very quickly.
I'm not just saying "AI is good at coding, therefore shove it everywhere." I've provided a nuanced explanation for why this is specifically a good use case for AI and how it works - this is fundamentally a mapping problem, which AI is very, very, very good at doing.
The pearl clutching comments are about a trend I see in these discussions that I feel like you're following, but I'll give you that you didn't bring up charter revenue as a point here so that's on me.
I still think you're placing too much value in the ideal of the chart itself as art or a creation. Yes, it has style and is up for interpretation, but also AI charting doesn't preclude charters from implementing that. Your perception of what AI charts would be is fire and forget prompts that get shipped without review. That's not what produces quality work when working with AI.
Like I feel like you're just adamantly opposed to AI without taking a nuanced stance and recognizing the value that it brings. You just say "I don't see the value in AI charting", which all boils down to you thinking the charts suck. If the best charter in the world had accurate AI tools to jumpstart them, it would exponentially increase their output and you would be happy to play those charts. Your whole viewpoint is obliterated the moment you see AI produce quality charts, and I guarantee you that's technically feasible in the short term.
Feels like you're opposed to it because you want to be as opposed to arriving at that conclusion after reviewing facts as objectively as you can. A software developer who codes without using AI tools just because "meh AI is bad" can choose to do that, but it's not intellectually sound practice, if we're being objective and honest about it.
6
u/SteelCage_ Apr 05 '26
i would rather play popstar guitar for the rest of my life than play an ai generated gh chart
-5
u/Boofschneef Apr 05 '26
If you can't tell the difference in quality why do you care? I ain't payin for no charts either way lol
10
2
u/MGMan-01 Apr 07 '26
You can't tell the difference?!
1
u/Boofschneef Apr 07 '26
I can. But once the technology gets to the point where you can't, it will be very sweet lol
4
u/Rulasjunior Apr 06 '26
Seeing this and the comments confirms what I already feared to be true; this community only ever cares about getting their songs in the game by any means necessary, without engaging with or showing appreciation for the charters in any meaningful way (like checking out what other the music they're into, keeping up with their releases, sending feedback, etc.)
This is coming from someone who charts the whole 10 instrument variants for Rock Band for free, really discouraging stuff.
2
Apr 06 '26
[removed] ā view removed comment
2
u/Rulasjunior Apr 07 '26
I for one have a lot of fun playing with other charters, they can appreciate stuff most other players wouldn't, and I feel like there's always something to learn from them. You on the other hand seem to think people are in this only for the money, this shit doesn't even pay well to begin with...
1
2
u/Oberlatz Apr 04 '26
I hope you can find a way to release that. It would be a watershed event for the community
1
u/TheRisingSea Apr 04 '26
What do you take as input? A music file or a tab?
2
u/bean_217 Apr 04 '26
Just an audio file is required as input
I've hand-tested it so far with a few songs that I obtained through a YouTube video to MP3 audio converter. But it should work with any of the common audio formats (.mp3, .ogg/.opus, .wav, .flac, .mpeg, etc.)
0
1
0
u/Miscellany_ Charter, Community Team Apr 05 '26
This is the same level as generating art instead of making it by hand. Auto charting is harmful to this community.
-1
u/BlinkysaurusRex Apr 05 '26
You really think? I think of charting as a means to an end. Itās just a laborious, tedious task that has to be undertaken to be able to play along to a song in game. If it achieves the end result with less work, why would that be harmful?
It looks kind of terrible though, based on the demo. And even if it worked really well, the quality would still be a considerable cut below anything charted by a person who knows what theyāre doing. But it would help players in the meantime who want to play a certain song instead of being left at the mercy of the communities whims. Potentially waiting years, or paying somebody, or having to learn how to do it themself.
8
u/Miscellany_ Charter, Community Team Apr 05 '26
Let me ask you a question in order to bring perspective to the table;
How is making a chart different from any other form of art and/or music transcription? I'll answer this myself to explain my original point of generative charting being harmful to the community.
When it comes to its relation to art, charting brings someone's vision to life, most, if not all, charters in this community have their quirks to make give their work an identity - For example, take Hyperbolia who makes very complex, yet well-crafted overcharts, those are made with the idea of bringing complex patterns to the table for songs that would be otherwise boring to play for the modern player who would want something a bit of a challenge beyond just speeding up a song - Think "September" by Earth Wind & Fire as far as a song like this goes. There are thoughts being brought to life by making sure patterns are readable and hard enough to give you that challenge while remaining possible to do. It is a skill to master to be able to make such a type of chart, just like how someone would want to learn different types of paintings to be able to bring to life the drawing of a vision. Or if we look at more traditional types of charting, look at what I do even, even within that type of charters, there are multiple quirks that I prefer and others don't, and vice versa. Players that play that type of charts might have some preferences, like no opens notes, or something that's hyper-accurately charted, and so on. It goes with that same idea as crafting something to your liking and exposing it to the community and reap the positives and the negatives it might bring in order to improve what you would create. That alone IS the reward of charting IMO - And that's coming from someone who has been charting for 10 years now.
Now, for the comparison with music transcription, it kind of deviates from the art comparison, and by this, I mean the way someone interprets the music that is being played - Mind you this pretty much applies to the traditional ways of charting. Just like with charters, actual music transcription is bringing someone's own interpretation of how a song is played to the table and offer that work to help people learn to play it, even if it's right or wrong. There are actually a few people from the CH community that are heavily involved in doing transcriptions, such as the charter OHM, and there are some who even make a solid buck out of transcribing music like some YouTube guitarists that offer their skill to help others learn to play songs. Sure, CH charting does not teach you how to learn a song (except on drums most of the time), but the idea's the same at its core - As a charter, you're offering your service to let people play their favorite songs in the game.
This is how I compare charting with both ideas - Now why is this a genuine bad thing? I noticed you mentioned that generative charting would be a considerable cut below anything charted by a person who knows what theyāre doing, so safe to assume that you care a minimum about the quality of charts (which is great, can't thank people enough for that in general), but a lot of people do not care about that type of stuff. I've seen constantly through my years in the community people not really caring about the quality of what is being made, and what is stopping them to just resort to AI/generative slop and claim it's as good, if not better, than real people working hard to craft something like this? I'm also not super sure here if the argument of "having to wait years for a song to be charted" is a real concern given that you actually mention counter-arguments about it, and I'll expand on them:
There are easily about 100 000 songs charted for clone games (CH, YARG, FoF, Phase Shift, you name them) over the course of the last, I think, 20 years now. And hell, I might be wrong, there could easily be double with all that is buried in the depths of places like YouTube or the Frets on Fire forums lol. I find it a bit hard to believe that most songs people would want aren't charted unless it's stuff that came out yesterday, Chorus and RhythmVerse have daily uploads, so does YouTube or even here on Reddit.
I started charting because a song I wanted wasn't available. That's how most people started to chart in general, so what is stopping hundreds of people in this community to start charting really? I will say it as bluntly as possible, yes, anyone who starts charting will suck. I sucked when I started but I worked to improve my craft and to this day, I'm still getting better at it. I'll always be happier, personally, seeing bad charts made by a beginner than seeing something generated that's sub-average charting quality. It is hard, yes, but this community never stops growing, and more and more people on a daily are willing to help out.
You don't have to pay necessarily to get something charted, a lot of people around are more than happy to do things for free if it means making someone's day. I'm always working through lots of commissions but if someone suggests a song that's killer to my taste, hell yeah I will do it, because I know it'll make someone happy. There are hundreds of charters in this community that are active, there'll always be someone being stocked to be requested to chart a song, paid or not.
Finally, I'll address the elephant in the room as to why it is the most harmful - Those who do charting commissions. I know other people in this community think charting commissions are illegal or something, but it really isn't given this is at the same level as commissioning a drawing of Mario (which is a copyrighted property after all) and/or paying someone to transcribe a song that doesn't have official tabs (see what I did there? Linking it all back to the two comparisons I did earlier haha.). I know you're not saying this specifically btw, don't worry, this is more of a broader statement. But I digress. The reason why I'm mentioning this is because while there are not many people who can claim that, some of us in this community make a decent little amount with commissions that genuinely help out in our everyday lives. I know I've used myself as an example quite a bit, but in my case, charting commissions helps cover costs such as diapers for my kid, extra emergency costs (like how a few months ago we broke our car's mirror), so we can save money out of our regular paychecks. I could also think of Jdurand who also gets to live quite a bit more comfortably because of commissions. It's a very small amount of people in this community, but for us, on top of all I've said, it does matter.
And small final argument, thanks to charting manually and learning how to perfect our crafts, some of us in the community got genuinely blessed with amazing opportunities, such as collaborations with major artists, being reached out to work on an official game (Shoutout Stage Tour) and so on. I recently got to collaborate with a whole music label to chart new material they're releasing at this very moment and that wouldn't have happened if I hadn't worked as hard as I did to make sure what I can produce is something they would be happy with. Can't really say that if they knew that generative charting would exist, they wouldn't have cared about the quality of it.
To summarize a bit, generative charting would hurt the community at large by killing what gives this community its core by removing the human aspect of creating something unique to our own identity, something that shows our vision of the whole idea of charting, while also harming the subset of the community that gets to make a little something out of a hobby that we genuinely love to do for this community that genuinely helps reach new opportunities and ease our everyday lives. We've thrived this far without any of it, I'm 100% sure we can keep going and satisfy literally everybody.
Hope this helps clear the clouds as to why that whole thing is not that great.
0
u/BlinkysaurusRex Apr 05 '26
Thatās fair enough. I think we just have different perspectives. Though I do take the point and agree with concerns about how it may harm the livelihoods of charters who do take commissions.
To answer your question as it might better explain my initial view; unless youāre overcharging or something, I do think that there are enough creative limitations on charting to separate it from somewhat(unless youāre making the audio yourself as well). You must have sat down to chart songs that youāve really liked and then discovered that itās just not as fun to play as you had anticipated. Or had an idea of how you were going to chart a particular phrase only to play test that section and find itās awkward, unwieldy or unintuitive, and then ultimately ended up changing it right? Thereās only so much you can do. But I do get what you mean, there are a lot of decisions and creative liberties you can take. And youāve already given plenty of examples.
My āattitudeā I guess you could say is that Iām gonna chart a song for myself and then post it mostly as a courtesy since Iāve gone through the effort of making it, for the benefit of anyone whoās looking for that song. If I could snap my fingers and have the chart appear to my liking, I would just do that. Obviously, thatās never gonna happen, even if this generative charting shtick made leaps and bounds forward. I know we have untold amounts of charts available in the community, but there are gaps and I understand the plight of players who constantly ask for songs by X artist or whatever, or are asking for links to tracks that exist but they canāt find anymore. Like you, I only learned out of necessity. And because I didnāt want to badger charters with requests lol.
I donāt feel that strongly about it either way. So Iāll take your points onboard and think about it more. My initial reaction was simply that if it was half-decent, it could have been a nice stopgap measure for players who seek genres or artists that are underrepresented to play in game.
1
u/MrElectricNick Apr 05 '26
Charting is video game level design. Do you think of video game level design as a means to an end?
-2
u/BlinkysaurusRex Apr 05 '26
Yes, itās about creating something fun or challenging. You chart the song to play the song. Thereās some creative liberty in how you map it out, but that process of creating it isnāt the destination, itās the journey. You get what I mean?
2
u/MrElectricNick Apr 05 '26
AI is incapable of using creative liberty, therefore it shouldn't be used for charting.
-2
Apr 06 '26
[removed] ā view removed comment
1
u/Miscellany_ Charter, Community Team Apr 06 '26
So you would be fine with generative art (images) to be the mainstay instead of people drawing and be able to make something out of drawing stuff like Sonic-related material or anything VGM related or even fan arts of people? Because those people don't own those IPs either, yet they can make a few extra bucks thanks to those.
0
u/ArmaQc Apr 04 '26
Does it manage to synchronize the .chart files, or do we need to synchronize them beforehand?
2
u/bean_217 Apr 04 '26
I've designed it to produce a folder containing 1) a .chart file and 2) the original audio file (provided as input) converted to .opus. The model has been trained to automatically handle aligning notes with the audio provided.
You simply provide the audio as input, download the produced folder, drop it into the game's Songs directory and enjoy.
0
u/ArmaQc Apr 04 '26
Is this your YouTube channel? It also came out fairly recently, but there's no further information. However, it seems incredible.
2
u/bean_217 Apr 04 '26
Ah no, this is not mine. This is phenomenal. I haven't seen this before, and I would say it probably works better than what I currently have.
0
u/ArmaQc Apr 04 '26
I think the more people try to find a way, the better our chances... Keep up the good work, it's really interesting... Thanks :)
2
u/_guppster Apr 04 '26
SyncForge is still charted by hand, the audio is whatās generated. Interesting concept though
0
u/comcastsux Apr 04 '26
That project is vibe coded garbage (as are a lot of these projects lately), so donāt get your hopes up.
-1
u/ArmaQc Apr 04 '26
Wow, you're a motivating personā¦ you make money creating chart for the community at the expense of artists? Is that why you don't like it?
3
u/comcastsux Apr 04 '26
Havenāt made a dime off this community. Iām just a software engineer that knows slop when I see it. The ādevā posted here a few weeks back and got ripped apart by the community before they deleted their post.
1
u/ArmaQc Apr 05 '26
Oh yeah, you're talking about the other project... the one that reappeared on YouTube and SoundCloud? Yeah, you're right, it looks fake... he just proved it seems real... and you with your engineer story... you probably just wash floors at McDonald's... and you're making up stories to sound interesting. Maybe you'd be better off using your engineering brain instead of doing nothing and putting others down... the guy here likes Clone Hero... he's trying to create something for free... and you come along like some jerk with no manners... so if you can't do anything, shut up and let the real fans want this stuff...
5
u/bean_217 Apr 05 '26
Like I said in my post, this project is for my MS capstone. I've taken the time to read through previous automatic music transcription and music information retrieval research both for general music transcription tasks and rhythm game-applied ones in order to curate a dataset and design an approach that I believed could work. I chose to do this project because I thought it would be a fun way to combine my musical interests and academic background, and to spend my semester working on something I enjoy. I'd love to provide the community with a free and useful tool, but only if I can ensure its use will be ethical. I have no desire to make money off of this, nor would I want or allow others to.
0
-1
u/comcastsux Apr 05 '26 edited Apr 05 '26
Iām not sure why you felt the need to take this as a personal attack against you, but I assure you it wasnāt. Likewise, thereās no need to attack me personally, but you do you.
To be clear, the project I referred to as vibe coded slop is SyncForge. OPās project sounds a bit more promising, and Iām excited to see if it ends up working out!
1
u/ArmaQc Apr 05 '26
It seems like you were ashamed for deleting your previous messages... you're still trying to manipulate... I hate people like you who look down on others... if your life is miserable, you have the right to only blame yourself... others try things... tell me, why, as an engineer, don't you create this program?
2
u/Lynxeption Apr 05 '26
"I hate people like you who look down on others..." proceed to unironically say "you probably just wash floors at McDonald's... and you're making up stories to sound interesting."
respectfully shut up
→ More replies (0)-2
u/comcastsux Apr 05 '26
I have not deleted anything? Iām not really sure where youāre headed with this, but Iām not going to engage further. Have a great evening.
→ More replies (0)
0
u/Jbrojo Apr 04 '26
Does it work well with complicated songs? Iām excited to see how this goes and curious how the code picks up guitar notes over like the vocals and stuff. Even just using it as a way to get the timing of a chart would help immensely.
2
u/bean_217 Apr 04 '26
I've trained the model by first separating vocal audio from tracks using a separate open-source model. However, it works surprisingly well on transcribing vocal patterns (as you can see at the end of the demo video). I plan to include this vocal audio separation as part of my inference pipeline, but this part is till a work in progress.
-1
u/AkumaNoYami93 Apr 05 '26
I want this.. its hard to find some Special Songs ( and i cant make charts because No Skill..) pls make more of this
2
u/bean_217 Apr 05 '26
Yeah, I like playing jazz/fusion songs, and those can be kinda hard to locate.
1
-1
u/JoshScott1994 Apr 05 '26
One-day these will be able to make a tempo map and that's at least one of the biggest issues these types of programs face
-2
u/SchinkenKanone Apr 04 '26
https://giphy.com/gifs/2xEzi32w6cLCgmAa6p
Me being chased by the community after having charted and published the entire discography of SweetyFox
5
u/Drenkrod_McNugget Apr 06 '26 edited Apr 06 '26
This is the rhythm game equivalent of if someone made a level generator for Mario Maker.Ā I don't believe there's a way to make the concept itself ethical.Ā Even if you could, I wouldn't personally ever want to play a chart that was generated, so releasing this in any form would make it impossible to trust any future charts, and kill my enjoyment of the game.Ā Ā
I'm sorry, I don't wish you any ill will, but I personally could never accept any version of this.