From ecade958af7912313064c299ecfc05545c510818 Mon Sep 17 00:00:00 2001 From: John Terpstra Date: Wed, 23 Apr 2003 23:22:48 +0000 Subject: More updates. (This used to be commit e4012c90323f7e87c3174d74020c6fcddb05820c) --- docs/docbook/projdoc/VFS.sgml | 25 ++- docs/docbook/projdoc/locking.sgml | 440 +++++++++++++++++++++++++++++++++----- 2 files changed, 409 insertions(+), 56 deletions(-) diff --git a/docs/docbook/projdoc/VFS.sgml b/docs/docbook/projdoc/VFS.sgml index 0a88543c6e..666eb4f62f 100644 --- a/docs/docbook/projdoc/VFS.sgml +++ b/docs/docbook/projdoc/VFS.sgml @@ -72,11 +72,28 @@ facility. The following operations are logged: This module is identical with the audit module above except that it sends audit logs to both syslog as well as the smbd log file/s. The -loglevel for this module is set in the smb.conf file. At loglevel = 0, only file -and directory deletions and directory and file creations are logged. At loglevel = 1 -file opens are renames and permission changes are logged , while at loglevel = 2 file -open and close calls are logged also. +loglevel for this module is set in the smb.conf file. + + +The logging information that will be written to the smbd log file is controlled by +the log level parameter in smb.conf. The +following information will be recorded: + + +Extended Auditing Log Information + + Log LevelLog Details - File and Directory Operations + + + 0Creation / Deletion + 1Create / Delete / Rename / Permission Changes + 2Create / Delete / Rename / Perm Change / Open / Close + + +
+ diff --git a/docs/docbook/projdoc/locking.sgml b/docs/docbook/projdoc/locking.sgml index ef65c16e2c..facaef551f 100644 --- a/docs/docbook/projdoc/locking.sgml +++ b/docs/docbook/projdoc/locking.sgml @@ -2,59 +2,395 @@ &author.jeremy; &author.jelmer; + &author.jht; +File and Record Locking -Locking - -One area which sometimes causes trouble is locking. - -There are two types of locking which need to be -performed by a SMB server. The first is "record locking" -which allows a client to lock a range of bytes in a open file. -The second is the "deny modes" that are specified when a file -is open. - -Record locking semantics under Unix is very -different from record locking under Windows. Versions -of Samba before 2.2 have tried to use the native -fcntl() unix system call to implement proper record -locking between different Samba clients. This can not -be fully correct due to several reasons. The simplest -is the fact that a Windows client is allowed to lock a -byte range up to 2^32 or 2^64, depending on the client -OS. The unix locking only supports byte ranges up to -2^31. So it is not possible to correctly satisfy a -lock request above 2^31. There are many more -differences, too many to be listed here. - -Samba 2.2 and above implements record locking -completely independent of the underlying unix -system. If a byte range lock that the client requests -happens to fall into the range 0-2^31, Samba hands -this request down to the Unix system. All other locks -can not be seen by unix anyway. - -Strictly a SMB server should check for locks before -every read and write call on a file. Unfortunately with the -way fcntl() works this can be slow and may overstress the -rpc.lockd. It is also almost always unnecessary as clients -are supposed to independently make locking calls before reads -and writes anyway if locking is important to them. By default -Samba only makes locking calls when explicitly asked -to by a client, but if you set "strict locking = yes" then it will -make lock checking calls on every read and write. - -You can also disable by range locking completely -using "locking = no". This is useful for those shares that -don't support locking or don't need it (such as cdroms). In -this case Samba fakes the return codes of locking calls to -tell clients that everything is OK. - -The second class of locking is the "deny modes". These -are set by an application when it opens a file to determine -what types of access should be allowed simultaneously with -its open. A client may ask for DENY_NONE, DENY_READ, DENY_WRITE -or DENY_ALL. There are also special compatibility modes called -DENY_FCB and DENY_DOS. + +Discussion + +One area which sometimes causes trouble is locking. + + + +There are two types of locking which need to be performed by a SMB server. +The first is record locking which allows a client to lock +a range of bytes in a open file. The second is the deny modes +that are specified when a file is open. + + + +Record locking semantics under Unix is very different from record locking under +Windows. Versions of Samba before 2.2 have tried to use the native fcntl() unix +system call to implement proper record locking between different Samba clients. +This can not be fully correct due to several reasons. The simplest is the fact +that a Windows client is allowed to lock a byte range up to 2^32 or 2^64, +depending on the client OS. The unix locking only supports byte ranges up to 2^31. +So it is not possible to correctly satisfy a lock request above 2^31. There are +many more differences, too many to be listed here. + + + +Samba 2.2 and above implements record locking completely independent of the +underlying unix system. If a byte range lock that the client requests happens +to fall into the range 0-2^31, Samba hands this request down to the Unix system. +All other locks can not be seen by unix anyway. + + + +Strictly a SMB server should check for locks before every read and write call on +a file. Unfortunately with the way fcntl() works this can be slow and may overstress +the rpc.lockd. It is also almost always unnecessary as clients are supposed to +independently make locking calls before reads and writes anyway if locking is +important to them. By default Samba only makes locking calls when explicitly asked +to by a client, but if you set strict locking = yes then it +will make lock checking calls on every read and write. + + + +You can also disable by range locking completely using locking = no. +This is useful for those shares that don't support locking or don't need it +(such as cdroms). In this case Samba fakes the return codes of locking calls to +tell clients that everything is OK. + + + +The second class of locking is the deny modes. These +are set by an application when it opens a file to determine what types of +access should be allowed simultaneously with its open. A client may ask for +DENY_NONE, DENY_READ, DENY_WRITE or DENY_ALL. There are also special compatibility +modes called DENY_FCB and DENY_DOS. + + + + +Samba Opportunistic Locking Control + + +Opportunistic locking essentially means that the client is allowed to download and cache +a file on their hard drive while making changes; if a second client wants to access the +file, the first client receives a break and must synchronise the file back to the server. +This can give significant performance gains in some cases; some programs insist on +synchronising the contents of the entire file back to the server for a single change. + + + +Level1 Oplocks (aka just plain "oplocks") is another term for opportunistic locking. + + + +Level2 Oplocks provids opportunistic locking for a file that will be treated as +read only. Typically this is used on files that are read-only or +on files that the client has no initial intention to write to at time of opening the file. + + + +Kernel Oplocks are essentially a method that allows the Linux kernel to co-exist with +Samba's oplocked files, although this has provided better integration of MS Windows network +file locking with the under lying OS, SGI IRIX and Linux are the only two OS's that are +oplock aware at this time. + + + +Unless your system supports kernel oplocks, you should disable oplocks if you are +accessing the same files from both Unix/Linux and SMB clients. Regardless, oplocks should +always be disabled if you are sharing a database file (e.g., Microsoft Access) between +multiple clients, as any break the first client receives will affect synchronisation of +the entire file (not just the single record), which will result in a noticable performance +impairment and, more likely, problems accessing the database in the first place. Notably, +Microsoft Outlook's personal folders (*.pst) react very badly to oplocks. If in doubt, +disable oplocks and tune your system from that point. + + + +If client-side caching is desirable and reliable on your network, you will benefit from +turning on oplocks. If your network is slow and/or unreliable, or you are sharing your +files among other file sharing mechanisms (e.g., NFS) or across a WAN, or multiple people +will be accessing the same files frequently, you probably will not benefit from the overhead +of your client sending oplock breaks and will instead want to disable oplocks for the share. + + + +Another factor to consider is the perceived performance of file access. If oplocks provide no +measurable speed benefit on your network, it might not be worth the hassle of dealing with them. + + + +You can disable oplocks on a per-share basis with the following: + + + oplocks = False + level2 oplocks = False + + +Alternately, you could disable oplocks on a per-file basis within the share: + + + veto oplock files = /*.mdb/*.MDB/*.dbf/*.DBF/ + + + + +If you are experiencing problems with oplocks as apparent from Samba's log entries, +you may want to play it safe and disable oplocks and level2 oplocks. + + + + + +MS Windows Opportunistic Locking and Caching Controls + + +There is a known issue when running applications (like Norton Anti-Virus) on a Windows 2000/ XP +workstation computer that can affect any application attempting to access shared database files +across a network. This is a result of a default setting configured in the Windows 2000/XP +operating system known as Opportunistic Locking. When a workstation +attempts to access shared data files located on another Windows 2000/XP computer, +the Windows 2000/XP operating system will attempt to increase performance by locking the +files and caching information locally. When this occurs, the application is unable to +properly function, which results in an Access Denied + error message being displayed during network operations. + + + +All Windows operating systems in the NT family that act as database servers for data files +(meaning that data files are stored there and accessed by other Windows PCs) may need to +have opportunistic locking disabled in order to minimize the risk of data file corruption. +This includes Windows 9x/Me, Windows NT, Windows 200x and Windows XP. + + + +If you are using a Windows NT family workstation in place of a server, you must also +disable opportunistic locking (oplocks) on that workstation. For example, if you use a +PC with the Windows NT Workstation operating system instead of Windows NT Server, and you +have data files located on it that are accessed from other Windows PCs, you may need to +disable oplocks on that system. + + + +The major difference is the location in the Windows registry where the values for disabling +oplocks are entered. Instead of the LanManServer location, the LanManWorkstation location +may be used. + + + +You can verify (or change or add, if necessary) this Registry value using the Windows +Registry Editor. When you change this registry value, you will have to reboot the PC +to ensure that the new setting goes into effect. + + + +The location of the client registry entry for opportunistic locking has changed in +Windows 2000 from the earlier location in Microsoft Windows NT. + + + +Windows 2000 will still respect the EnableOplocks registry value used to disable oplocks +in earlier versions of Windows. + + + +You can also deny the granting of opportunistic locks by changing the following registry entries: + + + + + HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MRXSmb\Parameters\ + + OplocksDisabled REG_DWORD 0 or 1 + Default: 0 (not disabled) + + + + +The OplocksDisabled registry value configures Windows clients to either request or not +request opportunistic locks on a remote file. To disable oplocks, the value of + OplocksDisabled must be set to 1. + + + + + HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters + + EnableOplocks REG_DWORD 0 or 1 + Default: 1 (Enabled by Default) + + EnableOpLockForceClose REG_DWORD 0 or 1 + Default: 0 (Disabled by Default) + + + + +The EnableOplocks value configures Windows-based servers (including Workstations sharing +files) to allow or deny opportunistic locks on local files. + + + +To force closure of open oplocks on close or program exit EnableOpLockForceClose must be set to 1. + + + +An illustration of how level II oplocks work: + + + + + Station 1 opens the file, requesting oplock. + + + Since no other station has the file open, the server grants station 1 exclusive oplock. + + + Station 2 opens the file, requesting oplock. + + + Since station 1 has not yet written to the file, the server asks station 1 to Break + to Level II Oplock. + + + Station 1 complies by flushing locally buffered lock information to the server. + + + Station 1 informs the server that it has Broken to Level II Oplock (alternatively, + station 1 could have closed the file). + + + The server responds to station 2's open request, granting it level II oplock. + Other stations can likewise open the file and obtain level II oplock. + + + Station 2 (or any station that has the file open) sends a write request SMB. + The server returns the write response. + + + The server asks all stations that have the file open to Break to None, meaning no + station holds any oplock on the file. Because the workstations can have no cached + writes or locks at this point, they need not respond to the break-to-none advisory; + all they need do is invalidate locally cashed read-ahead data. + + + + +Workstation Service Entries + + + \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters + + UseOpportunisticLocking REG_DWORD 0 or 1 + Default: 1 (true) + + + +Indicates whether the redirector should use opportunistic-locking (oplock) performance +enhancement. This parameter should be disabled only to isolate problems. + + + + +Server Service Entries + + + \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters + + EnableOplocks REG_DWORD 0 or 1 + Default: 1 (true) + + + +Specifies whether the server allows clients to use oplocks on files. Oplocks are a +significant performance enhancement, but have the potential to cause lost cached +data on some networks, particularly wide-area networks. + + + + MinLinkThroughput REG_DWORD 0 to infinite bytes per second + Default: 0 + + + +Specifies the minimum link throughput allowed by the server before it disables +raw and opportunistic locks for this connection. + + + + MaxLinkDelay REG_DWORD 0 to 100,000 seconds + Default: 60 + + + +Specifies the maximum time allowed for a link delay. If delays exceed this number, +the server disables raw I/O and opportunistic locking for this connection. + + + + OplockBreakWait REG_DWORD 10 to 180 seconds + Default: 35 + + + +Specifies the time that the server waits for a client to respond to an oplock break +request. Smaller values can allow detection of crashed clients more quickly but can +potentially cause loss of cached data. + + + + + + +Persistent Data Corruption + + +If you have applied all of the settings discussed in this paper but data corruption problems +and other symptoms persist, here are some additional things to check out: + + + +We have credible reports from developers that faulty network hardware, such as a single +faulty network card, can cause symptoms similar to read caching and data corruption. +If you see persistent data corruption even after repeated reindexing, you may have to +rebuild the data files in question. This involves creating a new data file with the +same definition as the file to be rebuilt and transferring the data from the old file +to the new one. There are several known methods for doing this that can be found in +our Knowledge Base. + + + + + +Additional Reading + + +You may want to check for an updated version of this white paper on our Web site from +time to time. Many of our white papers are updated as information changes. For those papers, +the Last Edited date is always at the top of the paper. + + + +Section of the Microsoft MSDN Library on opportunistic locking: + + + +Opportunistic Locks, Microsoft Developer Network (MSDN), Windows Development > +Windows Base Services > Files and I/O > SDK Documentation > File Storage > File Systems +> About File Systems > Opportunistic Locks, Microsoft Corporation. +http://msdn.microsoft.com/library/en-us/fileio/storage_5yk3.asp + + + +Microsoft Knowledge Base Article Q224992 "Maintaining Transactional Integrity with OPLOCKS", +Microsoft Corporation, April 1999, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q224992. + + + +Microsoft Knowledge Base Article Q296264 "Configuring Opportunistic Locking in Windows 2000", +Microsoft Corporation, April 2001, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q296264. + + + +Microsoft Knowledge Base Article Q129202 "PC Ext: Explanation of Opportunistic Locking on Windows NT", + Microsoft Corporation, April 1995, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q129202. + + + -- cgit