r2159: converted samba4 over to UTF-16.

I had previously thought this was unnecessary, as windows doesn't use
standards compliant UTF-16, and for filesystem operations treats bytes
as UCS-2, but Bjoern Jacke has pointed out to me that this means we
don't correctly store extended UTF-16 characters as UTF-8 on
disk. This can be seen with (for example) the gothic characters with
codepoints above 64k.

This commit also adds a LOCAL-ICONV torture test that tests the first
1 million codepoints against the system iconv library, and tests 5
million random UTF-16LE buffers for identical error handling to the
system iconv library.

the lib/iconv.c changes need backporting to samba3
(This used to be commit 756f28ac95feaa84b42402723d5f7286865c78db)

1 files changed, 2 insertions, 2 deletions

diff --git a/source4/lib/util_unistr.c b/source4/lib/util_unistr.c
index 2bd990836e..63d68fa12e 100644
--- a/source4/lib/util_unistr.c
+++ b/source4/lib/util_unistr.c
@@ -96,9 +96,9 @@ static int check_dos_char(smb_ucs2_t c)
 	char buf[10];
 	smb_ucs2_t c2 = 0;
 	int len1, len2;
-	len1 = convert_string(CH_UCS2, CH_DOS, &c, 2, buf, sizeof(buf));
+	len1 = convert_string(CH_UTF16, CH_DOS, &c, 2, buf, sizeof(buf));
 	if (len1 == 0) return 0;
-	len2 = convert_string(CH_DOS, CH_UCS2, buf, len1, &c2, 2);
+	len2 = convert_string(CH_DOS, CH_UTF16, buf, len1, &c2, 2);
 	if (len2 != 2) return 0;
 	return (c == c2);
 }