diff options
author | John Terpstra <jht@samba.org> | 2005-06-16 01:33:35 +0000 |
---|---|---|
committer | Gerald W. Carter <jerry@samba.org> | 2008-04-23 08:46:49 -0500 |
commit | fa96398866a4bcdcc13b42ab4f8d3f516cd9238a (patch) | |
tree | ca055132ca3289d5b512b8cc3858033be3df3bae /docs/Samba3-HOWTO/TOSHARG-Unicode.xml | |
parent | 77aa4181f19460a6e8b848877edb107c09f574d8 (diff) | |
download | samba-fa96398866a4bcdcc13b42ab4f8d3f516cd9238a.tar.gz samba-fa96398866a4bcdcc13b42ab4f8d3f516cd9238a.tar.bz2 samba-fa96398866a4bcdcc13b42ab4f8d3f516cd9238a.zip |
Stage 1 of PHPTR Edits.
(This used to be commit 64a9e3e8619bf33dcf6b0ff8171b47a3e2581239)
Diffstat (limited to 'docs/Samba3-HOWTO/TOSHARG-Unicode.xml')
-rw-r--r-- | docs/Samba3-HOWTO/TOSHARG-Unicode.xml | 226 |
1 files changed, 113 insertions, 113 deletions
diff --git a/docs/Samba3-HOWTO/TOSHARG-Unicode.xml b/docs/Samba3-HOWTO/TOSHARG-Unicode.xml index 761116c818..c1d8fc1611 100644 --- a/docs/Samba3-HOWTO/TOSHARG-Unicode.xml +++ b/docs/Samba3-HOWTO/TOSHARG-Unicode.xml @@ -22,8 +22,8 @@ <para> Every industry eventually matures. One of the great areas of maturation is in the focus that has been given over the past decade to make it possible for anyone -anywhere to use a computer. It has not always been that way, in fact, not so long -ago it was common for software to be written for exclusive use in the country of +anywhere to use a computer. It has not always been that way. In fact, not so long +ago, it was common for software to be written for exclusive use in the country of origin. </para> @@ -36,8 +36,8 @@ is deserving of special mention. <para> Samba-2.x supported a single locale through a mechanism called -<emphasis>codepages</emphasis>. Samba-3 is destined to become a truly trans-global -file and printer-sharing platform. +<emphasis>codepages</emphasis>. Samba-3 is destined to become a truly transglobal +file- and printer-sharing platform. </para> </sect1> @@ -46,7 +46,7 @@ file and printer-sharing platform. <title>What Are Charsets and Unicode?</title> <para> -Computers communicate in numbers. In texts, each number will be +Computers communicate in numbers. In texts, each number is translated to a corresponding letter. The meaning that will be assigned to a certain number depends on the <emphasis>character set (charset) </emphasis> that is used. @@ -58,21 +58,21 @@ letters. Not all computers use the same charset (there are charsets with German umlauts, Japanese characters, and so on). The American Standard Code for Information Interchange (ASCII) encoding system has been the normative character encoding scheme used by computers to date. This employs a charset that contains -256 characters. Using this mode of encoding each character takes exactly one byte. +256 characters. Using this mode of encoding, each character takes exactly one byte. </para> <para> There are also charsets that support extended characters, but those need at least twice as much storage space as does ASCII encoding. Such charsets can contain <command>256 * 256 = 65536</command> characters, which is more than all possible -characters one could think of. They are called multi-byte charsets because they use +characters one could think of. They are called multibyte charsets because they use more then one byte to store one character. </para> <para> -One standardized multi-byte charset encoding scheme is known as +One standardized multibyte charset encoding scheme is known as <ulink url="http://www.unicode.org/">unicode</ulink>. A big advantage of using a -multi-byte charset is that you only need one. There is no need to make sure two +multibyte charset is that you only need one. There is no need to make sure two computers use the same charset when they are communicating. </para> @@ -80,7 +80,7 @@ computers use the same charset when they are communicating. <parameter>codepages</parameter>, by Microsoft. However, there is no support for negotiating the charset to be used in the SMB/CIFS protocol. Thus, you have to make sure you are using the same charset when talking to an older client. -Newer clients (Windows NT, 200x, XP) talk unicode over the wire. +Newer clients (Windows NT, 200x, XP) talk Unicode over the wire. </para> </sect1> @@ -88,7 +88,7 @@ Newer clients (Windows NT, 200x, XP) talk unicode over the wire. <title>Samba and Charsets</title> <para> -As of Samba-3, Samba can (and will) talk unicode over the wire. Internally, +As of Samba-3, Samba can (and will) talk Unicode over the wire. Internally, Samba knows of three kinds of character sets: </para> @@ -98,15 +98,15 @@ Samba knows of three kinds of character sets: <listitem><para> This is the charset used internally by your operating system. The default is <constant>UTF-8</constant>, which is fine for most - systems, which covers all characters in all languages. The default + systems and covers all characters in all languages. The default in previous Samba releases was to save filenames in the encoding of the - clients, for example cp850 for western european countries. + clients &smbmdash; for example, cp850 for Western European countries. </para></listitem> </varlistentry> <varlistentry> <term><smbconfoption name="display charset"/></term> - <listitem><para>This is the charset Samba will use to print messages + <listitem><para>This is the charset Samba uses to print messages on your screen. It should generally be the same as the <parameter>unix charset</parameter>. </para></listitem> </varlistentry> @@ -114,7 +114,7 @@ Samba knows of three kinds of character sets: <varlistentry> <term><smbconfoption name="dos charset"/></term> <listitem><para>This is the charset Samba uses when communicating with - DOS and Windows 9x/Me clients. It will talk unicode to all newer clients. + DOS and Windows 9x/Me clients. It will talk Unicode to all newer clients. The default depends on the charsets you have installed on your system. Run <command>testparm -v | grep "dos charset"</command> to see what the default is on your system. @@ -152,29 +152,29 @@ Setting up Japanese charsets is quite difficult. This is mainly because: <listitem><para> Mainly for historical reasons, there are several encoding methods in Japanese, which are not fully compatible with each other. There are - two major encoding methods. One is the Shift_JIS series, it is used in Windows - and some UNIX's. The other is the EUC-JP series, used in most UNIX's + two major encoding methods. One is the Shift_JIS series used in Windows + and some UNIXes. The other is the EUC-JP series used in most UNIXes and Linux. Moreover, Samba previously also offered several unique encoding methods, named CAP and HEX, to keep interoperability with CAP/NetAtalk and - UNIX's which can't use Japanese filenames. Some implementations of the + UNIXes that can't use Japanese filenames. Some implementations of the EUC-JP series can't support the full Windows character set. </para></listitem> <listitem><para>There are some code conversion tables between Unicode and legacy Japanese character sets. One is compatible with Windows, another one - is based on the reference of the Unicode consortium and others are + is based on the reference of the Unicode consortium, and others are a mixed implementation. The Unicode consortium does not officially define any conversion tables between Unicode and legacy character - sets so there cannot be standard one. + sets, so there cannot be standard one. </para></listitem> - <listitem><para>The character set and conversion tables available in iconv() depends + <listitem><para>The character set and conversion tables available in iconv() depend on the iconv library that is available. Next to that, the Japanese locale names may be different on different systems. This means that the value of the charset parameters depends on the implementation of iconv() you are using. </para> - <para>Though 2 byte fixed UCS-2 encoding is used in Windows internally, + <para>Though 2-byte fixed UCS-2 encoding is used in Windows internally, Shift_JIS series encoding is usually used in Japanese environments as ASCII encoding is in English environments. </para></listitem> @@ -183,7 +183,7 @@ Setting up Japanese charsets is quite difficult. This is mainly because: <sect2><title>Basic Parameter Setting</title> <para> - <smbconfoption name="dos charset"/> and + The <smbconfoption name="dos charset"/> and <smbconfoption name="display charset"/> should be set to the locale compatible with the character set and encoding method used on Windows. This is usually CP932 @@ -191,13 +191,13 @@ Setting up Japanese charsets is quite difficult. This is mainly because: </para> <para> - <smbconfoption name="unix charset"/> can be either Shift_JIS series, - EUC-JP series and UTF-8. UTF-8 is always available but the availability of other locales - and its name itself depends on the system. + The <smbconfoption name="unix charset"/> can be either Shift_JIS series, + EUC-JP series, or UTF-8. UTF-8 is always available, but the availability of other locales + and the name itself depends on the system. </para> <para> - Additionally, you can consider to use the Shift_JIS series as the + Additionally, you can consider using the Shift_JIS series as the value of the <smbconfoption name="unix charset"/> parameter by using the vfs_cap module, which does the same thing as setting <quote>coding system = CAP</quote> in the Samba 2.2 series. @@ -205,40 +205,40 @@ Setting up Japanese charsets is quite difficult. This is mainly because: <para> Where to set <smbconfoption name="unix charset"/> - to is a difficult question. Here is a list of details, advantages and + to is a difficult question. Here is a list of details, advantages, and disadvantages of using a certain value. </para> <variablelist> <varlistentry><term>Shift_JIS series</term> <listitem><para> - Shift_JIS series means a locale which is equivalent to <constant>Shift_JIS</constant>, + Shift_JIS series means a locale that is equivalent to <constant>Shift_JIS</constant>, used as a standard on Japanese Windows. In the case of <constant>Shift_JIS</constant>, - for example if a Japanese file name consist of 0x8ba4 and 0x974c - (a 4 bytes Japanese character string meaning <quote>share</quote>) and <quote>.txt</quote> - is written from Windows on Samba, the file name on UNIX becomes - 0x8ba4, 0x974c, <quote>.txt</quote> (a 8 bytes BINARY string), same as Windows. + for example, if a Japanese filename consists of 0x8ba4 and 0x974c + (a 4-bytes Japanese character string meaning <quote>share</quote>) and <quote>.txt</quote> + is written from Windows on Samba, the filename on UNIX becomes + 0x8ba4, 0x974c, <quote>.txt</quote> (an 8-byte BINARY string), same as Windows. </para> - <para>Since Shift_JIS series is usually used on some commercial based - UNIX's; hp-ux and AIX as Japanese locale (however, it is also possible - to use the EUC-JP series), To use Shift_JIS series on these platforms, - Japanese file names created from Windows can be referred to also on + <para>Since Shift_JIS series is usually used on some commercial-based + UNIXes; hp-ux and AIX as the Japanese locale (however, it is also possible + to use the EUC-JP locale series). To use Shift_JIS series on these platforms, + Japanese filenames created from Windows can be referred to also on UNIX.</para> <para> If your UNIX is already working with Shift_JIS and there is a user - who needs to use Japanese file names written from Windows, the - Shift_JIS series is the best choice. However, broken file names - may be displayed and some commands which cannot handle non-ASCII - filenames may be aborted during parsing filenames. especially there - may be <quote>\ (0x5c)</quote> in file names, which need to be handled carefully. - So you had better not touch file names written from Windows on UNIX. + who needs to use Japanese filenames written from Windows, the + Shift_JIS series is the best choice. However, broken filenames + may be displayed, and some commands that cannot handle non-ASCII + filenames may be aborted during parsing filenames. Especially, there + may be <quote>\ (0x5c)</quote> in filenames, which need to be handled carefully. + It is best to not touch filenames written from Windows on UNIX. </para> <para> Note that most Japanized free software actually works with EUC-JP - only. You had better verify if the Japanized free software can work + only. It is good practice to verify that the Japanized free software can work with Shift_JIS. </para> </listitem> @@ -246,58 +246,51 @@ Setting up Japanese charsets is quite difficult. This is mainly because: <varlistentry><term>EUC-JP series</term> <listitem><para> - EUC-JP series means a locale which is equivalent to the industry + EUC-JP series means a locale that is equivalent to the industry standard called EUC-JP, widely used in Japanese UNIX (although EUC contains specifications for languages other than Japanese, such as - EUC-KR). In the case of EUC-JP series, for example if a Japanese - file name consist of 0x8ba4 and 0x974c and <quote>.txt</quote> is written from - Windows on Samba, the file name on UNIX becomes 0xb6a6, 0xcdad, - <quote>.txt</quote> (a 8 bytes BINARY string). + EUC-KR). In the case of EUC-JP series, for example, if a Japanese + filename consists of 0x8ba4 and 0x974c and <quote>.txt</quote> is written from + Windows on Samba, the filename on UNIX becomes 0xb6a6, 0xcdad, + <quote>.txt</quote> (an 8-byte BINARY string). </para> <para> - Since EUC-JP is usually used on Open source UNIX, Linux and FreeBSD, - and on commercial based UNIX, Solaris, IRIX and Tru64 UNIX as - Japanese locale (however, it is also possible on Solaris to use - Shift_JIS and UTF-8, on Tru64 UNIX to use Shift_JIS). To use EUC-JP - series, most Japanese file names created from Windows can be - referred to also on UNIX. Also, most Japanized free software work - mainly with EUC-JP only. + Since EUC-JP is usually used on open source UNIX, Linux, and FreeBSD, and on commercial-based UNIX, Solaris, + IRIX, and Tru64 UNIX as Japanese locale (however, it is also possible on Solaris to use Shift_JIS and UTF-8, + and on Tru64 UNIX it is possible to use Shift_JIS). To use EUC-JP series, most Japanese filenames created from + Windows can be referred to also on UNIX. Also, most Japanized free software work mainly with EUC-JP only. </para> <para> - It is recommended to choose EUC-JP series when using Japanese file - names on these UNIX. + It is recommended to choose EUC-JP series when using Japanese filenames on UNIX. </para> <para> - Although there is no character which needs to be carefully treated - like <quote>\ (0x5c)</quote>, broken file names may be displayed and some - commands which cannot handle non-ASCII filenames may be aborted + Although there is no character that needs to be carefully treated + like <quote>\ (0x5c)</quote>, broken filenames may be displayed and some + commands that cannot handle non-ASCII filenames may be aborted during parsing filenames. </para> <para> Moreover, if you built Samba using differently installed libiconv, - eucJP-ms locale included in libiconv and EUC-JP series locale - included in OS may not be compatible. In this case, you may need to - avoid using incompatible characters for file names. + the eucJP-ms locale included in libiconv and EUC-JP series locale + included in the operating system may not be compatible. In this case, you may need to + avoid using incompatible characters for filenames. </para> </listitem> </varlistentry> <varlistentry><term>UTF-8</term> <listitem><para> - UTF-8 means a locale which is equivalent to UTF-8, the international - standard defined by Unicode consortium. In UTF-8, a <parameter>character</parameter> is - expressed using 1-3 bytes. In case of Japanese, most characters - are expressed using 3 bytes. Since on Windows Shift_JIS, where a - character is expressed with 1 or 2 bytes, is used to express - Japanese, basically a byte length of a UTF-8 string grows 1.5 times - the length of a original Shift_JIS string. In the case of UTF-8, - for example if a Japanese file name consist of 0x8ba4 and 0x974c and - <quote>.txt</quote> is written from Windows on Samba, the file name on UNIX - becomes 0xe585, 0xb1e6, 0x9c89, <quote>.txt</quote> (a 10 bytes BINARY string). + UTF-8 means a locale equivalent to UTF-8, the international standard defined by the Unicode consortium. In + UTF-8, a <parameter>character</parameter> is expressed using 1 to 3 bytes. In case of the Japanese language, + most characters are expressed using 3 bytes. Since on Windows Shift_JIS, where a character is expressed with 1 + or 2 bytes is used to express Japanese, basically a byte length of a UTF-8 string the length of the UTF-8 + string is 1.5 times that of the original Shift_JIS string. In the case of UTF-8, for example, if a Japanese + filename consists of 0x8ba4 and 0x974c, and <quote>.txt</quote> is written from Windows on Samba, the filename + on UNIX becomes 0xe585, 0xb1e6, 0x9c89, <quote>.txt</quote> (a 10-byte BINARY string). </para> <para> @@ -306,28 +299,29 @@ Setting up Japanese charsets is quite difficult. This is mainly because: </para> <para> - There are no systems that use UTF-8 as default locale for Japanese. + There are no systems that use UTF-8 as the default locale for Japanese. </para> <para> - Some broken file names may be displayed and some commands which + Some broken filenames may be displayed, and some commands that cannot handle non-ASCII filenames may be aborted during parsing - filenames. especially there may be <quote>\ (0x5c)</quote> in file names, which - need to be handled carefully. So you had better not touch file names + filenames. Especially, there may be <quote>\ (0x5c)</quote> in filenames, which + must be handled carefully, so you had better not touch filenames written from Windows on UNIX. </para> <para> In addition, although it is not directly concerned with Samba, since - there is a delicate difference between iconv() function, which is - generally used on UNIX and the functions used on other platforms, - such as Windows and Java about the conversion table between - Shift_JIS and Unicode, you should be carefully to handle UTF-8. + there is a delicate difference between the iconv() function, which is + generally used on UNIX, and the functions used on other platforms, + such as Windows and Java, so far is concerens the conversion between + Shift_JIS and Unicode UTF-8 must be done with care and recognition + of the limitations involved in the process. </para> <para> Although Mac OS X uses UTF-8 as its encoding method for filenames, - it uses an extended UTF-8 specification that Samba cannot handle so + it uses an extended UTF-8 specification that Samba cannot handle, so UTF-8 locale is not available for Mac OS X. </para> </listitem> @@ -335,43 +329,44 @@ Setting up Japanese charsets is quite difficult. This is mainly because: <varlistentry><term>Shift_JIS series + vfs_cap (CAP encoding)</term> <listitem><para> - CAP encoding means a specification using in CAP and NetAtalk, file + CAP encoding means a specification used in CAP and NetAtalk, file server software for Macintosh. In the case of CAP encoding, for - example if a Japanese file name consist of 0x8ba4 and 0x974c and - <quote>.txt</quote> is written from Windows on Samba, the file name on UNIX + example, if a Japanese filename consists of 0x8ba4 and 0x974c, and + <quote>.txt</quote> is written from Windows on Samba, the filename on UNIX becomes <quote>:8b:a4:97L.txt</quote> (a 14 bytes ASCII string). </para> <para> - For CAP encoding a byte which cannot be expressed as an ASCII - character (0x80 or above) is encoded as <quote>:xx</quote> form. You need to take - care of containing a <quote>\(0x5c)</quote> in a filename but filenames are not - broken in a system which cannot handle non-ASCII filenames. + For CAP encoding, a byte that cannot be expressed as an ASCII + character (0x80 or above) is encoded in an <quote>:xx</quote> form. You need to take + care of containing a <quote>\(0x5c)</quote> in a filename, but filenames are not + broken in a system that cannot handle non-ASCII filenames. </para> <para> The greatest merit of CAP encoding is the compatibility of encoding - filenames with CAP or NetAtalk, file server software of Macintosh. - Since they usually write a file name on UNIX with CAP encoding, if a + filenames with CAP or NetAtalk. These are respectively the Columbia Appletalk + Protocol, and the NetAtalk Open Source software project. + Since these software applications write a file name on UNIX with CAP encoding, if a directory is shared with both Samba and NetAtalk, you need to use - CAP encoding to avoid non-ASCII filenames are broken. + CAP encoding to avoid non-ASCII filenames from being broken. </para> <para> - However, recently there are some systems where NetAtalk has been - patched to write filenames with EUC-JP (i.e. Japanese original Vine Linux). - Here you need to choose EUC-JP series instead of CAP encoding. + However, recently, NetAtalk has been + patched on some systems to write filenames with EUC-JP (e.g., Japanese original Vine Linux). + In this case, you need to choose EUC-JP series instead of CAP encoding. </para> <para> - vfs_cap itself is available for non Shift_JIS series locales for - systems which cannot handle non-ASCII characters or systems which - shares files with NetAtalk. + vfs_cap itself is available for non-Shift_JIS series locales for + systems that cannot handle non-ASCII characters or systems that + share files with NetAtalk. </para> <para> To use CAP encoding on Samba-3, you should use the unix charset parameter and VFS - as follows: + as in Example 29.5.1: </para> <example><title>VFS CAP</title> @@ -387,7 +382,7 @@ Setting up Japanese charsets is quite difficult. This is mainly because: </example> <para> - You should set CP932 if using GNU libiconv for unix charset. Setting this, + You should set CP932 if using GNU libiconv for unix charset. With this setting, filenames in the <quote>cap-share</quote> share are written with CAP encoding. </para> </listitem> @@ -426,8 +421,8 @@ display charset = CP932 </programlisting> <para> - Other Japanese locales (for example Shift_JIS and EUC-JP) should not - be used for the lack of the compatibility with Windows. + Other Japanese locales (for example, Shift_JIS and EUC-JP) should not + be used because of the lack of the compatibility with Windows. </para> </listitem> </varlistentry> @@ -449,8 +444,8 @@ display charset = CP932 </smbconfblock> <para> - Other Japanese locales (for example Shift_JIS and EUC-JP) should not - be used for the lack of the compatibility with Windows. + Other Japanese locales (for example, Shift_JIS and EUC-JP) should not + be used because of the lack of the compatibility with Windows. </para> </listitem> </varlistentry> @@ -462,9 +457,10 @@ display charset = CP932 <title>Migration from Samba-2.2 Series</title> <para> -Prior to Samba-2.2 series <quote>coding system</quote> parameter is used as -<smbconfoption name="unix charset"/> parameter of the Samba-3 series. -<link linkend="japancharsets">Next table</link> shows the mapping table when migrating from the Samba-2.2 series to Samba-3. +Prior to Samba-2.2 series, the <quote>coding system</quote> parameter was used. The default codepage in Samba +2.x was code page 850. In the Samba-3 series this has been replaced with the <smbconfoption name="unix +charset"/> parameter. <link linkend="japancharsets">Japanese Character Sets in Samba-2.2 and Samba-3</link> +shows the mapping table when migrating from the Samba-2.2 series to Samba-3. </para> <table frame="all" id="japancharsets"> @@ -501,12 +497,16 @@ Prior to Samba-2.2 series <quote>coding system</quote> parameter is used as <para><quote>Samba is complaining about a missing <filename>CP850.so</filename> file.</quote></para> - <para><emphasis>Answer:</emphasis> CP850 is the default <smbconfoption name="dos charset"/>. - The <smbconfoption name="dos charset"/> is used to convert data to the codepage used by your dos clients. - If you do not have any dos clients, you can safely ignore this message. </para> + <para> + CP850 is the default <smbconfoption name="dos charset"/>. + The <smbconfoption name="dos charset"/> is used to convert data to the codepage used by your DOS clients. + If you do not have any DOS clients, you can safely ignore this message. </para> - <para>CP850 should be supported by your local iconv implementation. Make sure you have all the required packages installed. - If you compiled Samba from source, make sure to configure found iconv.</para> + <para> + CP850 should be supported by your local iconv implementation. Make sure you have all the required packages installed. + If you compiled Samba from source, make sure that the configure process found iconv. This can be + confirmed by checking the <filename>config.log</filename> file that is generated when + <command>configure</command> is executed.</para> </sect2> </sect1> |