Moving docs tree to docs-xml to make room for generated docs in the release tarball.

(This used to be commit 9f672c26d63955f613088489c6efbdc08b5b2d14)

1 files changed, 89 insertions, 0 deletions

diff --git a/docs-xml/Samba3-HOWTO/TOSHARG-LargeFile.xml b/docs-xml/Samba3-HOWTO/TOSHARG-LargeFile.xml
new file mode 100644
index 0000000000..5fdbe7a243
--- /dev/null
+++ b/docs-xml/Samba3-HOWTO/TOSHARG-LargeFile.xml
@@ -0,0 +1,89 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!DOCTYPE chapter PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
+<chapter id="largefile">
+<chapterinfo>
+	&author.jeremy;
+	&author.jht;
+	<pubdate>March 5, 2005</pubdate>
+</chapterinfo>
+<title>Handling Large Directories</title>
+
+<para>
+<indexterm><primary>performance degradation</primary></indexterm>
+<indexterm><primary>large numbers of files</primary></indexterm>
+<indexterm><primary>large directory</primary></indexterm>
+Samba-3.0.12 and later implements a solution for sites that have experienced performance degradation due to the
+problem of using Samba-3 with applications that need large numbers of files (100,000 or more) per directory.
+</para>
+
+<para>
+<indexterm><primary>read directory into memory</primary></indexterm>
+<indexterm><primary>strange delete semantics</primary></indexterm>
+The key was fixing the directory handling to read only the current list requested instead of the old
+(up to samba-3.0.11) behavior of reading the entire directory into memory before doling out names.
+Normally this would have broken OS/2 applications, which have very strange delete semantics, but by
+stealing logic from Samba4 (thanks, Tridge), the current code in 3.0.12 handles this correctly.
+</para>
+
+<para>
+<indexterm><primary>large directory</primary></indexterm>
+<indexterm><primary>performance</primary></indexterm>
+To set up an application that needs large numbers of files per directory in a way that does not
+damage performance unduly, follow these steps:
+</para>
+
+<para>
+<indexterm><primary>canonicalize files</primary></indexterm>
+First, you need to canonicalize all the files in the directory to have one case, upper or lower &smbmdash; take your
+pick (I chose upper because all my files were already uppercase names). Then set up a new custom share for the
+application as follows:
+<smbconfblock>
+<smbconfsection name="[bigshare]"/>
+<smbconfoption name="path">/data/manyfilesdir</smbconfoption>
+<smbconfoption name="read only">no</smbconfoption>
+<smbconfoption name="case sensitive">True</smbconfoption>
+<smbconfoption name="default case">upper</smbconfoption>
+<smbconfoption name="preserve case">no</smbconfoption>
+<smbconfoption name="short preserve case">no</smbconfoption>
+</smbconfblock>
+</para>
+
+<para>
+<indexterm><primary>case options</primary></indexterm>
+<indexterm><primary>match case</primary></indexterm>
+<indexterm><primary>uppercase</primary></indexterm>
+Of course, use your own path and settings, but set the case options to match the case of all the files in your
+directory. The path should point at the large directory needed for the application &smbmdash; any new files created in
+there and in any paths under it will be forced by smbd into uppercase, but smbd will no longer have to scan
+the directory for names: it knows that if a file does not exist in uppercase, then it doesn't exist at all.
+</para>
+
+<para>
+<indexterm><primary>case-insensitive</primary></indexterm>
+<indexterm><primary>consistent case</primary></indexterm>
+<indexterm><primary>smbd</primary></indexterm>
+The secret to this is really in the <smbconfoption name="case sensitive">True</smbconfoption>
+line. This tells smbd never to scan for case-insensitive versions of names. So if an application asks for a file
+called <filename>FOO</filename>, and it cannot be found by a simple stat call, then smbd will return file not
+found immediately without scanning the containing directory for a version of a different case. The other
+<filename>xxx case xxx</filename> lines make this work by forcing a consistent case on all files created by
+&smbd;.
+</para>
+
+<para>
+<indexterm><primary>uppercase</primary></indexterm>
+<indexterm><primary>stanza</primary></indexterm>
+<indexterm><primary>lowercase filenames</primary></indexterm>
+Remember, all files and directories under the <parameter>path</parameter> directory must be in uppercase
+with this &smb.conf; stanza because &smbd; will not be able to find lowercase filenames with these settings. Also
+note that this is done on a per-share basis, allowing this parameter to be set only for a share servicing an application with
+this problematic behavior (using large numbers of entries in a directory) &smbmdash; the rest of your &smbd; shares
+don't need to be affected.
+</para>
+
+<para>
+This makes smbd much faster when dealing with large directories.  My test case has over 100,000 files, and
+smbd now deals with this very efficiently.
+</para>
+
+</chapter>