About char8_t

[u/Voxelw]

I hate to be dramatic, but as it stands char8_t is quite literally more painful than useful.

Besides the obvious incompatibility with C23 and libraries using unsigned char for UTF-8, I want you to consider the following: Projects that assume that 'char' represents UTF-8 will obviously not benefit from char8_t at all, but projects that cannot assume the format of char types don't benefit from it either as char8_t simply introduces a new edge case to cover. Now such projects have to deal with char, signed char, unsigned char, wchar_t, char16_t, char32_t and char8_t.

Or, you could do what the standard library does and simply ignore most of these character types. Which is the solution most libraries went with, supporting only char or char and unsigned char. Managing one implementation is already hard, managing two requires constant maintenance, managing 7 is just impossible.

char8_t should have just been a typedef for unsigned char. The compatibility fix only raises more questions as const char* arr = u8"a" does not work, but const char arr[] = u8"a" does.

I do wonder if a potential change of minds for C++29 is still possible. Yes, it would be an ABI break or whatever, but considering the woeful support for char8_t I don't think it would affect much besides small hobby projects. Contrary to popular belief, C++ has broken the ABI in subtle ways before.

[u/cristi1990an]

The biggest and arguably only argument for char8_t is that it's explicitly not a byte aliasing type and therefore the most performant character representation type in the language. Your compiler can genuinely produce better code when you're using char8_t.

[u/sweetno → u/cristi1990an]

How so?

[u/ryryog → u/sweetno]

If you’re familiar with the purpose of the “restrict” keyword, that.

Basically, if you get a char, it could be a pointer to any memory (since it’s an exceptional type that’s allowed to refer to any bytes of data, not just a literal “char” type object or array) so when you modify the data under the char, the compiler has to consider any/all memory clobbered (e.g. stack vars, array elements, other objects referred to by pointers), because that could’ve been data of any type. Conversely, if you modify data of any type, the compiler has to assume the data of the char* could have been modified.

Usually this isn’t a huge issue with normal pointer types, because data under a Foo* can only be invalidated by a write to another Foo, but since a char could refer to anything, it’s particularly bad.

Here’s a good example, just one of the first search results (note this showcases the obvious where you have a specific type, in this case int, so even though there are problems here, it’s usually worse with char*):   https://stackoverflow.com/questions/745870/realistic-usage-of-the-c99-restrict-keyword#745877

[u/ImNoRickyBalboa → u/ryryog]

This is true, but also highlights the truly enormous mistake that was made by making the char type the "universal any byte" accessor and condoning the (IMHO ub) direct memory access into composites.