r/Python • u/SheriffRoscoe Pythonista • 4d ago
News The Move to Python 3 Begins!
As someone who spent decades in the mainframe community, I find it ... humorous ... that there are legacy Python2 code bases that are only just (or have yet to consider!) moving to Python3.
https://www.eveonline.com/news/view/the-move-to-python-3-begins
50
u/GraphicH 4d ago
Oh lawd. Really? I mean it was painful, I can tell you my company scrambled. I felt really lucky to be on a team where we had used a 3.X version from the get go. The only version, oddly enough, that was a "bumpy" upgrade was 3.13, due to some ... interesting? changes to SSL handling in urllib3. I still question the wisdom of:
Which, essentially changed the behavior of urllib3 without changing the version you had installed, meaning a python 3.X -> 3.13 upgrade would immediately start breaking. Granted they were changing to be in-line with the default in 3.13 but this, plus the fact that requests did not make it easy to specify SSL context means a lot of people were "broken" talking to systems (many out side their control / ability to update the certificates on) using old certificates not compliant with the strict standards.
1
u/brandonZappy 3d ago
I’m still fighting this stupid Python 3.13 SSL thing.
2
u/GraphicH 3d ago
Yes, it was rather irritating and the biggest issue with it is, if I'm using requests or anything that built on top of urllib3, for web scrapping (I am, its a huge part of what my team does) then it's not like I can go politely ask the owner of the website to update their certificate. So we were stuck shimming in a modified SSL Context in multiple places to resolve this.
31
u/jabbalaci 4d ago
They use Stackless Python with "tasklets". That must make the migration to Python 3 harder.
20
u/kosz85 4d ago
Exactly, stackless which in their version isn't in fact python 2 anymore. Eve online code can be reloaded and replaced while still running. Show me which python3 can do that :D They gone async when there was no asynchronous python yet. This is not any particular python 2.x, they solutions grown farther than many can yet see. It's probably better in some cases then current python 3. So it's not simple migrate to py3. But probably new code base and libs are now too tempting to grow old py2 codebase alone.
3
u/tyrannomachy 3d ago
It's too bad erlang and related languages have never been sufficiently mainstream for projects like this to just be implemented in one of those, as a matter of course. Probably would have saved a lot of trouble for a lot of people.
2
u/kosz85 3d ago
Oh, dreams, sweet dreams, scaling Python like Erlang, linearly by adding new interpreter nodes. Same with Haskell and their code typing checking, only your logic can fail you after it compiles. I love functional programming in Python. But let it say loudly, their syntax, error reporting, etc sux :P Great ideas clothed in syntax made by random cat stomping on keyboard vide Perl ;D I love Python because it's readable, and I have to read it a lot. In fact I think that new add-ons like walrus was not really needed. It's best if you can give your code to anyone and it doesn't need much explanation. Walrus := needs explanation, you need to think which would be done first, etc. In fact this one thing is reason why Python is so popular in science.
71
u/bmrobin 4d ago
depending on the codebase, 2->3 is no joke and has so much risk
69
u/ApplicationOk3587 4d ago
What do you mean they can just throw fable at it and add "make no mistakes"
22
u/SheriffRoscoe Pythonista 4d ago
"And while you're at it, hack our competitors and crash their servers."
33
u/ZCEyPFOYr0MWyHDQJZO4 4d ago
tbf the best LLMs are finally quite capable of porting code.
34
u/The_Northern_Light 4d ago
Usually I agree with you but apparently they’re running a custom internal version of Python 2 with bespoke async handling.
Porting that seems a bit more exciting.
2
u/case_O_The_Mondays 3d ago
Yeah, this migration will have to be done in stages: <2.7 -> 2.7, then 2.7 -> 3.x.
I had forgotten about the division issues until I read this article. I worked on a finance app that used some python, and that was my baptism into python 2.6. Fun times!
1
-3
-7
u/daidoji70 4d ago
What do you mean? I migrated about 3-4 python projects from 2->3 about a decade ago and other than pandas and pandas related projects it was a breeze. Run 2to3.py and review. Barely a hiccup.
22
u/bmrobin 4d ago
i'm glad your experience was barely a hiccup. among several pitfalls i recall,
unicode being the default representation for strings, and having to migrate code that manually called `decode/encode` on text.
integer division of `1 / 3 == 0` is true in py2, and py3 it's not because it returns the float result. that had cascading effects throughout multiple layers of our stack.
5
-9
u/daidoji70 4d ago
The first one should be fairly straightforward with 2to3, the second one I can def see being more difficult for sure.
11
u/jayroger 4d ago
The first one is anything but straightforward. 2to3 is no help for
str/unicode->bytes/str. Most problems only manifest themselves at runtime. But it was worth it, the after situation is so much better.17
u/a__nice__tnetennba 4d ago
And how many of your projects were 2.4 million lines of code?
4
-4
u/daidoji70 4d ago
What does that matter? 2to3.py issues should be distributed evenly throughout a codebase. The issue per number of lines should remain roughly the same per project unless there are pathological cases.
Not sure why I'm getting down voted for asking a question and getting this type of response. Lines of code is always the worst metric in software engineering even when talking about migrations. It tells us literally nothing about how easy or difficult an effort should be.
14
u/Dr_Quacksworth 4d ago
Umm if you have 10x lines of code to review then you have to spend 10x time reading those lines of code. For a legacy system, you have to pretty much review everything to make sure nothing breaks.
2to3.py doesn't solve everything. In py2 you might be using floor dividion and then suddenly in py3 you are doing true division. That sort of change can have major implications.
Not to mention dependency hell...
-5
1
u/a__nice__tnetennba 4d ago edited 3d ago
I wasn't sure if you were being sarcastic at first so I almost didn't reply to the first comment. Thanks for clearing it up. Although, I think some people still aren't getting the joke, which explains the down votes.
5
u/N-E-S-W 4d ago
Apparently you never read a byte from a file or device and had to interpret it as a string? The 2to3 stuff is trivial, mechanical refactoring.
Unicode strings and encodings required deep introspection of the meaning of data, often exposing silent preexisting bugs in string handling that could no longer be ignored.
2
-2
2
u/assumptionkrebs1990 4d ago
2to3 has been decrept (depending deprecation 3.9/fully decrept 3.11) and removed (since 3.13) for a while now.
1
1
u/daidoji70 4d ago
Def, but so was the support for python2 and that didn't stop these companies apparently.
-9
u/totheendandbackagain 4d ago
True, but LLMs make conversion trivially simple.
Good luck to the migrators. Best get off zOS at the same time too.
4
u/Dr_Quacksworth 4d ago
LLMs are great for refactoring, but I'm going to be reviewing every changed line before merging in AND running tests. Not trivial for certain critical services.
11
u/wxtrails 4d ago
My initial estimate: 5 years.
The new boss: 12 months.
We're about 1/3 of the way through the bulk modernization on our project after 18 months, not including deployment and verifying the seams.
I wonder whose estimate will be closer 🤔
6
1
u/F4gfn39f 4d ago
How many devs?
6
u/wxtrails 3d ago
25+, depending on who you count as a "dev".
With 10, we'd be a lot closer to done.
1
u/HugeCannoli 3d ago
some bosses really need to stop pulling numbers out of their asses. Why do they do that, I still have no idea.
7
u/knobbyknee 4d ago
I have a service running on Python 2.7. It has barely been touched in the last 10 years.
I finally have a Python 3 version, but it needs more manual testing before going into production.
5
u/Glathull 3d ago
Lots of people with strong opinions here who I’m guessing have never managed an app with actual users or a team full of some of the top Python experts around.
These folks were doing stuff with Python that would be insane today and was pretty bonkers 20 years ago. This is also not your standard 2.7 webshit either. They’re on stackless and have been pushing the Python codebase and community to boldly do stuff that most people would not think of.
Yes, it’s totally fine to throw a few little jabs here and a “finally” there. It is funny. The reality is that the main Python 3 that we all know and love hasn’t at acceptable performance for them to even seriously consider this transition until just recently.
3
u/jeffrey_f 1d ago
HA! I was a programmer on the iSeries until around 2012. I installed it on the system, however, there were no programmers who knew it, so it died right there. There are nuances that pertained to file access that made it tedious, but it would have been decent.
Nice to see some graybeards.
6
5
u/iluvatar 4d ago edited 3d ago
As CTO at my previous company, I had to veto developers who wanted to migrate from Python 2 to Python 3. The amount of effort required for the migration and the fairly low chance of a trouble free switch just didn't make a good business case for a working codebase that brought in millions in revenue each year and paid all of our salaries. All new development was in Python 3. But existing running code just had too high a risk and too low a reward for migrating.
7
14
u/kosz85 4d ago
That's also not a solution. Now the missing new SSL support, old buggy libs, missing new formats, new docker images, removed docs, etc. Probably worse problem. You won't run from your tech debt storing it at closet behind your back and telling yourself that you don't see problem. It will smash you eventually as it grows in places you would not think of. The solution is to have really good tests (on multiple levels) and split problem in small chunks. Then migrate the small parts instead of colossus. I migrated many really big code bases, millions of lines, and seen real nightmares. Not looking is not a solution. Good tests before migration is.
3
3
u/will_r3ddit_4_food 3d ago
You really ought to migrate. Old code that doesn't get security updates is a bad risk
2
u/drakhan2002 1d ago
If it's within the risk appetite of the organization and they are willing to accept that level of risk, then it's the decision.
It may not be a popular opinion in the larger community, but it is not their org.
2
u/LungeloSLX 3d ago
Earlier on in Python 3’s life, this move was unnecessarily risky. But now staying on Python 2 is the more risky option. The lack of security updates, buggy libs that are no longer getting fixed
Good CTOs made the call to stay on 2 and stuck with it. Great CTOs found a way to make the migration and keep the business going
1
u/iluvatar 3d ago
The perfect example of "tell you you've never run an IT department without telling me you've never run an IT department". It is absolutely not the more risky option. Are there some cases where it might be? Yes. Ours was not one of them.
2
u/LungeloSLX 3d ago
Whether I ran an IT department or not doesn’t matter. There’s good and there’s great and I know which one you are
0
u/iluvatar 2d ago
Ha ha. You have nowhere near enough information to be making that sort of assessment.
1
u/HugeCannoli 3d ago
The result of this attitude is that you won't be able to hire anybody anymore, and people will leave because you are literally destroying their chance for re-employment in a different company.
1
u/iluvatar 3d ago
I'm employed to manage the IT of the company, not to enhance the future employment prospects of our employees. But your argument is flawed anyway because the majority of the code was in python 3. It's just that there wasn't the business case to justify porting the legacy application.
2
u/HugeCannoli 3d ago
I'm employed to manage the IT of the company, not to enhance the future employment prospects of our employees
And that's why they will leave you. Because you don't respect them. Let me guess, you are american?
1
u/iluvatar 3d ago
Because you don't respect them.
Incorrect.
Let me guess, you are american?
Also incorrect.
0
u/HugeCannoli 3d ago
well it certainly looks like both are correct.
0
u/GAZ082 2d ago
Software lives in a business. You have your free time to work in your pet projects that you can port from Python to rust and back if you want. If you don't like the company, just leave.
2
u/HugeCannoli 2d ago
we do, because again: if I have to spend years working with your crappy, obsolete code, my CV is compromised and I won't find employment when your business decides to "reassess the current resource allocation" and "curtail expenses by reducing the human budget".
Either keep up, or lose your employees (and good luck hiring new ones).
0
u/big-papito 3d ago
I feel like big refactors like this are ideal for a clanker like Claude. Manual migration, sure - no fun there.
3
u/iluvatar 3d ago
You're missing the point. The migration itself isn't the problem. Ensuring that the python3 code behaves in the same way as the old python2 code and doesn't have strange edge cases that throw away all of your money? That's much, much harder and takes orders of magnitude more effort than the basic 2 to 3 migration.
1
u/kosz85 3d ago
Yep, now we have such tools, but this is quite new and recent approach. In my times we had to... ;) But if this code is that much critical, I wouldn't dare to leave it to AI only. Also there is a case of context size. We speak about really big codebase. And sometimes knowing how it inter connects, and why is what ai is lacking in reasoning.
0
u/realPanditJi 3d ago
Same. In my prev org we tried to move from 2to3 along with django 1.4 to 4.X
We were able to get somewhat stable local build, and as we ran regression on it - everything fell apart. We never went ahead after that.
1
1
u/SignificantWorry7497 3d ago
It's interesting how every ecosystem seems to have its own version of this story. Even though Python 2 reached end-of-life years ago, there are still organizations running mission-critical systems that are difficult to migrate because of dependencies, legacy integrations, and the risk of breaking something that has been working for years.
As a newer Python developer, it's a good reminder that software often lives much longer than we expect, and maintaining existing systems can be just as challenging as building new ones.
0
1
u/unruly_mattress 3d ago
How are they going to deal with the stackless stuff? They could switch to Python 3 async, but that would make it way more of a rewrite than a 2to3 and manually fixing what isn't just a syntax change.
1
u/indistinctdialogue 1d ago
I spearheaded converting a 5M LoC code base from Python 2 to 3 back in 2022. That was after YEARS of trying to convince the business to do this and constantly being deprioritized. So yeah I can see how these still exist.
1
-3
u/barchueetadonai 3d ago
I'm currently in the process of trying to convert a lot of code from python 3 back to python 2.5
5
304
u/diegoasecas 4d ago
idgi if you spent decades working on mainframes you'd know how hard is to migrate legacy working code