&author.jht;
StephenLangasekvorlon@netexpress.netMay 31, 2003PAM-Based Distributed AuthenticationPAM-enabledWinbindADSWinbind-based authentication
This chapter should help you to deploy Winbind-based authentication on any PAM-enabled
UNIX/Linux system. Winbind can be used to enable user-level application access authentication
from any MS Windows NT domain, MS Windows 200x Active Directory-based
domain, or any Samba-based domain environment. It will also help you to configure PAM-based local host access
controls that are appropriate to your Samba configuration.
PAM managementpam_smbpass.so
In addition to knowing how to configure Winbind into PAM, you will learn generic PAM management
possibilities and in particular how to deploy tools like pam_smbpass.so to your advantage.
The use of Winbind requires more than PAM configuration alone.
Please refer to Winbind: Use of Domain Accounts, for further information regarding Winbind.
Features and BenefitsSun SolarisxxxxBSDLinuxPluggable Authentication ModulesPAM/etc/passwdloginpasswdchown
A number of UNIX systems (e.g., Sun Solaris), as well as the xxxxBSD family and Linux,
now utilize the Pluggable Authentication Modules (PAM) facility to provide all authentication,
authorization, and resource control services. Prior to the introduction of PAM, a decision
to use an alternative to the system password database (/etc/passwd)
would require the provision of alternatives for all programs that provide security services.
Such a choice would involve provision of alternatives to programs such as login,
passwd, chown, and so on.
PAM/etc/pam.confSolaris/etc/pam.d
PAM provides a mechanism that disconnects these security programs from the underlying
authentication/authorization infrastructure. PAM is configured by making appropriate modifications to one file,
/etc/pam.conf (Solaris), or by editing individual control files that are
located in /etc/pam.d.
PAM-enableddynamically loadable library modules
On PAM-enabled UNIX/Linux systems, it is an easy matter to configure the system to use any
authentication backend so long as the appropriate dynamically loadable library modules
are available for it. The backend may be local to the system or may be centralized on a
remote server.
PAM support modules are available for:
/etc/passwd/etc/passwdPAM modulespam_unix.sopam_unix2.sopam_pwdb.sopam_userdb.so
There are several PAM modules that interact with this standard UNIX user database. The most common are called
pam_unix.so, pam_unix2.so, pam_pwdb.so and
pam_userdb.so.
Kerberospam_krb5.soKerberosHeimdalMIT KerberosADS
The pam_krb5.so module allows the use of any Kerberos-compliant server.
This tool is used to access MIT Kerberos, Heimdal Kerberos, and potentially
Microsoft Active Directory (if enabled).
LDAPLDAPpam_ldap.soOpenLDAPSun ONE iDentity serverNovell eDirectory serverMicrosoft Active Directory
The pam_ldap.so module allows the use of any LDAP v2- or v3-compatible backend
server. Commonly used LDAP backend servers include OpenLDAP v2.0 and v2.1,
Sun ONE iDentity server, Novell eDirectory server, and Microsoft Active Directory.
NetWare BinderyNetWare Binderypam_ncp_auth.sobindery-enabledNetWare Core Protocol-based server
The pam_ncp_auth.so module allows authentication off any bindery-enabled
NetWare Core Protocol-based server.
SMB PasswordSMB Passwordpam_smbpass.sopassdb backend
This module, called pam_smbpass.so, allows user authentication of
the passdb backend that is configured in the Samba &smb.conf; file.
SMB ServerSMB Serverpam_smb_auth.so
The pam_smb_auth.so module is the original MS Windows networking authentication
tool. This module has been somewhat outdated by the Winbind module.
WinbindWinbindpam_winbind.sodomain controllerauthentication
The pam_winbind.so module allows Samba to obtain authentication from any
MS Windows domain controller. It can just as easily be used to authenticate
users for access to any PAM-enabled application.
RADIUSRemote Access Dial-In User ServiceRADIUS
There is a PAM RADIUS (Remote Access Dial-In User Service) authentication
module. In most cases, administrators need to locate the source code
for this tool and compile and install it themselves. RADIUS protocols are
used by many routers and terminal servers.
pam_smbpasswd.sopam_winbind.so
Of the modules listed, Samba provides the pam_smbpasswd.so and the
pam_winbind.so modules alone.
wide-area network bandwidthefficient authenticationPAM-capablecentrally managed
Once configured, these permit a remarkable level of flexibility in the location and use
of distributed Samba domain controllers that can provide wide-area network bandwidth,
efficient authentication services for PAM-capable systems. In effect, this allows the
deployment of centrally managed and maintained distributed authentication from a
single-user account database.
Technical DiscussionPAMprivilege-granting applications/etc/pam.conf/etc/pam.d/
PAM is designed to provide system administrators with a great deal of flexibility in
configuration of the privilege-granting applications of their system. The local
configuration of system security controlled by PAM is contained in one of two places:
either the single system file /etc/pam.conf or the
/etc/pam.d/ directory.
PAM Configuration SyntaxPAM-specific tokenscase sensitivity
In this section we discuss the correct syntax of and generic options respected by entries to these files.
PAM-specific tokens in the configuration file are case insensitive. The module paths, however, are case
sensitive, since they indicate a file's name and reflect the case dependence of typical file systems. The
case sensitivity of the arguments to any given module is defined for each module in turn.
In addition to the lines described below, there are two special characters provided for the convenience
of the system administrator: comments are preceded by a # and extend to the next end-of-line; also,
module specification lines may be extended with a \-escaped newline.
PAM authentication module/lib/security
If the PAM authentication module (loadable link library file) is located in the
default location, then it is not necessary to specify the path. In the case of
Linux, the default location is /lib/security. If the module
is located outside the default, then the path must be specified as:
auth required /other_path/pam_strange_module.so
Anatomy of /etc/pam.d Entries
The remaining information in this subsection was taken from the documentation of the Linux-PAM
project. For more information on PAM, see
the Official Linux-PAM home page.
/etc/pam.conf
A general configuration line of the /etc/pam.conf file has the following form:
service-name module-type control-flag module-path args
We explain the meaning of each of these tokens. The second (and more recently adopted)
way of configuring Linux-PAM is via the contents of the /etc/pam.d/ directory.
Once we have explained the meaning of the tokens, we describe this method.
service-nameftpdrlogindsu
The name of the service associated with this entry. Frequently, the service-name is the conventional
name of the given application &smbmdash; for example, ftpd, rlogind and
su, and so on.
There is a special service-name reserved for defining a default authentication mechanism. It has
the name OTHER and may be specified in either lower- or uppercase characters.
Note, when there is a module specified for a named service, the OTHER
entries are ignored.
module-type
One of (currently) four types of module. The four types are as follows:
auth/etc/groupsauth: This module type provides two aspects of authenticating the user.
It establishes that the user is who he or she claims to be by instructing the application
to prompt the user for a password or other means of identification. Second, the module can
grant group membership (independently of the /etc/groups file)
or other privileges through its credential-granting properties.
accountnon-authentication-based account managementaccount: This module performs non-authentication-based account management.
It is typically used to restrict/permit access to a service based on the time of day, currently
available system resources (maximum number of users), or perhaps the location of the user
login. For example, the root login may be permitted only on the console.
sessionsession: Primarily, this module is associated with doing things that need
to be done for the user before and after he or she can be given service. Such things include logging
information concerning the opening and closing of some data exchange with a user, mounting
directories, and so on.
passwordpassword: This last module type is required for updating the authentication
token associated with the user. Typically, there is one module for each
challenge/response-based authentication (auth) module type.
control-flag
The control-flag is used to indicate how the PAM library will react to the success or failure of the
module it is associated with. Since modules can be stacked (modules of the same type execute in series,
one after another), the control-flags determine the relative importance of each module. The application
is not made aware of the individual success or failure of modules listed in the
/etc/pam.conf file. Instead, it receives a summary success or fail response from
the Linux-PAM library. The order of execution of these modules is that of the entries in the
/etc/pam.conf file; earlier entries are executed before later ones.
As of Linux-PAM v0.60, this control-flag can be defined with one of two syntaxes.
requiredrequisitesufficientoptional
The simpler (and historical) syntax for the control-flag is a single keyword defined to indicate the
severity of concern associated with the success or failure of a specific module. There are four such
keywords: required, requisite,
sufficient, and optional.
The Linux-PAM library interprets these keywords in the following manner:
required: This indicates that the success of the module is required for the
module-type facility to succeed. Failure of this module will not be apparent to the user until all
of the remaining modules (of the same module-type) have been executed.
requisite: Like required, except that if such a module returns a
failure, control is directly returned to the application. The return value is that associated with
the first required or requisite module to fail. This flag can be used to protect against the
possibility of a user getting the opportunity to enter a password over an unsafe medium. It is
conceivable that such behavior might inform an attacker of valid accounts on a system. This
possibility should be weighed against the not insignificant concerns of exposing a sensitive
password in a hostile environment.
sufficient: The success of this module is deemed sufficient to satisfy
the Linux-PAM library that this module-type has succeeded in its purpose. In the event that no
previous required module has failed, no more stacked modules of this type are invoked.
(In this case, subsequent required modules are not invoked). A failure of this module is not deemed
as fatal to satisfying the application that this module-type has succeeded.
optional: As its name suggests, this control-flag marks the module as not
being critical to the success or failure of the user's application for service. In general,
Linux-PAM ignores such a module when determining if the module stack will succeed or fail.
However, in the absence of any definite successes or failures of previous or subsequent stacked
modules, this module will determine the nature of the response to the application. One example of
this latter case is when the other modules return something like PAM_IGNORE.
The more elaborate (newer) syntax is much more specific and gives the administrator a great deal of control
over how the user is authenticated. This form of the control-flag is delimited with square brackets and
consists of a series of value=action tokens:
[value1=action1 value2=action2 ...]
Here, value1 is one of the following return values:
success; open_err; symbol_err; service_err; system_err; buf_err;perm_denied; auth_err; cred_insufficient; authinfo_unavail;user_unknown; maxtries; new_authtok_reqd; acct_expired; session_err;cred_unavail; cred_expired; cred_err; no_module_data; conv_err;authtok_err; authtok_recover_err; authtok_lock_busy;authtok_disable_aging; try_again; ignore; abort; authtok_expired;module_unknown; bad_item; and default.
The last of these (default) can be used to set the action for those return values that are not explicitly defined.
The action1 can be a positive integer or one of the following tokens:
ignore; ok; done;
bad; die; and reset.
A positive integer, J, when specified as the action, can be used to indicate that the next J modules of the
current module-type will be skipped. In this way, the administrator can develop a moderately sophisticated
stack of modules with a number of different paths of execution. Which path is taken can be determined by the
reactions of individual modules.
ignore: When used with a stack of modules, the module's return status will not
contribute to the return code the application obtains.
bad: This action indicates that the return code should be thought of as indicative
of the module failing. If this module is the first in the stack to fail, its status value will be used
for that of the whole stack.
die: Equivalent to bad with the side effect of terminating the module stack and
PAM immediately returning to the application.
ok: This tells PAM that the administrator thinks this return code should
contribute directly to the return code of the full stack of modules. In other words, if the former
state of the stack would lead to a return of PAM_SUCCESS, the module's return code will override
this value. Note, if the former state of the stack holds some value that is indicative of a module's
failure, this ok value will not be used to override that value.
done: Equivalent to ok with the side effect of terminating the module stack and
PAM immediately returning to the application.
reset: Clears all memory of the state of the module stack and starts again with
the next stacked module.
Each of the four keywords, required; requisite;
sufficient; and optional, have an equivalent expression in terms
of the [...] syntax. They are as follows:
required is equivalent to [success=ok new_authtok_reqd=ok ignore=ignore default=bad].
requisite is equivalent to [success=ok new_authtok_reqd=ok ignore=ignore default=die].
sufficient is equivalent to [success=done new_authtok_reqd=done default=ignore].
optional is equivalent to [success=ok new_authtok_reqd=ok default=ignore].
Just to get a feel for the power of this new syntax, here is a taste of what you can do with it. With Linux-PAM-0.63,
the notion of client plug-in agents was introduced. This makes it possible for PAM to support
machine-machine authentication using the transport protocol inherent to the client/server application. With the
[ ... value=action ... ] control syntax, it is possible for an application to be configured
to support binary prompts with compliant clients, but to gracefully fail over into an alternative authentication
mode for legacy applications.
module-path
The pathname of the dynamically loadable object file; the pluggable module itself. If the first character of the
module path is /, it is assumed to be a complete path. If this is not the case, the given module path is appended
to the default module path: /lib/security (but see the previous notes).
The arguments are a list of tokens that are passed to the module when it is invoked, much like arguments to a typical
Linux shell command. Generally, valid arguments are optional and are specific to any given module. Invalid arguments
are ignored by a module; however, when encountering an invalid argument, the module is required to write an error
to syslog(3). For a list of generic options, see the next section.
If you wish to include spaces in an argument, you should surround that argument with square brackets. For example:
squid auth required pam_mysql.so user=passwd_query passwd=mada \
db=eminence [query=select user_name from internet_service where \
user_name=%u and password=PASSWORD(%p) and service=web_proxy]
When using this convention, you can include [ characters inside the string, and if you wish to have a ]
character inside the string that will survive the argument parsing, you should use \[. In other words,
[..[..\]..] --> ..[..]..
Any line in one of the configuration files that is not formatted correctly will generally tend (erring on the
side of caution) to make the authentication process fail. A corresponding error is written to the system log files
with a call to syslog(3).
Example System Configurations
The following is an example /etc/pam.d/login configuration file.
This example had all options uncommented and is probably not usable
because it stacks many conditions before allowing successful completion
of the login process. Essentially, all conditions can be disabled
by commenting them out, except the calls to pam_pwdb.so.
PAM: Original Login Config
#%PAM-1.0
# The PAM configuration file for the login service
#
auth required pam_securetty.so
auth required pam_nologin.so
# auth required pam_dialup.so
# auth optional pam_mail.so
auth required pam_pwdb.so shadow md5
# account requisite pam_time.so
account required pam_pwdb.so
session required pam_pwdb.so
# session optional pam_lastlog.so
# password required pam_cracklib.so retry=3
password required pam_pwdb.so shadow md5
PAM: Login Using pam_smbpass
PAM allows use of replaceable modules. Those available on a sample system include:
$/bin/ls /lib/security
pam_access.so pam_ftp.so pam_limits.so
pam_ncp_auth.so pam_rhosts_auth.so pam_stress.so
pam_cracklib.so pam_group.so pam_listfile.so
pam_nologin.so pam_rootok.so pam_tally.so
pam_deny.so pam_issue.so pam_mail.so
pam_permit.so pam_securetty.so pam_time.so
pam_dialup.so pam_lastlog.so pam_mkhomedir.so
pam_pwdb.so pam_shells.so pam_unix.so
pam_env.so pam_ldap.so pam_motd.so
pam_radius.so pam_smbpass.so pam_unix_acct.so
pam_wheel.so pam_unix_auth.so pam_unix_passwd.so
pam_userdb.so pam_warn.so pam_unix_session.so
The following example for the login program replaces the use of
the pam_pwdb.so module that uses the system
password database (/etc/passwd,
/etc/shadow, /etc/group) with
the module pam_smbpass.so, which uses the Samba
database containing the Microsoft MD4 encrypted password
hashes. This database is stored either in
/usr/local/samba/private/smbpasswd,
/etc/samba/smbpasswd or in
/etc/samba.d/smbpasswd, depending on the
Samba implementation for your UNIX/Linux system. The
pam_smbpass.so module is provided by
Samba version 2.2.1 or later. It can be compiled by specifying the
options when running Samba's
configure script. For more information
on the pam_smbpass module, see the documentation
in the source/pam_smbpass directory of the Samba
source distribution.
#%PAM-1.0
# The PAM configuration file for the login service
#
auth required pam_smbpass.so nodelay
account required pam_smbpass.so nodelay
session required pam_smbpass.so nodelay
password required pam_smbpass.so nodelay
The following is the PAM configuration file for a particular
Linux system. The default condition uses pam_pwdb.so.
#%PAM-1.0
# The PAM configuration file for the samba service
#
auth required pam_pwdb.so nullok nodelay shadow audit
account required pam_pwdb.so audit nodelay
session required pam_pwdb.so nodelay
password required pam_pwdb.so shadow md5
In the following example, the decision has been made to use the
smbpasswd database even for basic Samba authentication. Such a
decision could also be made for the passwd program and would
thus allow the smbpasswd passwords to be changed using the
passwd program:
#%PAM-1.0
# The PAM configuration file for the samba service
#
auth required pam_smbpass.so nodelay
account required pam_pwdb.so audit nodelay
session required pam_pwdb.so nodelay
password required pam_smbpass.so nodelay smbconf=/etc/samba.d/smb.conf
PAM allows stacking of authentication mechanisms. It is
also possible to pass information obtained within one PAM module through
to the next module in the PAM stack. Please refer to the documentation for
your particular system implementation for details regarding the specific
capabilities of PAM in this environment. Some Linux implementations also
provide the pam_stack.so module that allows all
authentication to be configured in a single central file. The
pam_stack.so method has some devoted followers
on the basis that it allows for easier administration. As with all issues in
life, though, every decision has trade-offs, so you may want to examine the
PAM documentation for further helpful information.
&smb.conf; PAM Configuration
There is an option in &smb.conf; called .
The following is from the online help for this option in SWAT:
When Samba is configured to enable PAM support (i.e., ), this parameter will
control whether or not Samba should obey PAM's account and session management directives. The default behavior
is to use PAM for clear-text authentication only and to ignore any account or session management. Samba always
ignores PAM for authentication in the case of yes.
The reason is that PAM modules cannot support the challenge/response authentication mechanism needed in the presence of SMB
password encryption.
Default: no
Remote CIFS Authentication Using winbindd.so
All operating systems depend on the provision of user credentials acceptable to the platform.
UNIX requires the provision of a user identifier (UID) as well as a group identifier (GID).
These are both simple integer numbers that are obtained from a password backend such
as /etc/passwd.
Users and groups on a Windows NT server are assigned a relative ID (RID) which is unique for
the domain when the user or group is created. To convert the Windows NT user or group into
a UNIX user or group, a mapping between RIDs and UNIX user and group IDs is required. This
is one of the jobs that winbind performs.
As winbind users and groups are resolved from a server, user and group IDs are allocated
from a specified range. This is done on a first come, first served basis, although all
existing users and groups will be mapped as soon as a client performs a user or group
enumeration command. The allocated UNIX IDs are stored in a database file under the Samba
lock directory and will be remembered.
The astute administrator will realize from this that the combination of pam_smbpass.so,
winbindd, and a distributed
such as ldap will allow the establishment of a centrally managed, distributed user/password
database that can also be used by all PAM-aware (e.g., Linux) programs and applications. This arrangement can have
particularly potent advantages compared with the use of Microsoft Active Directory Service (ADS) insofar as
the reduction of wide-area network authentication traffic.
The RID to UNIX ID database is the only location where the user and group mappings are
stored by winbindd. If this file is deleted or corrupted, there is no way for winbindd
to determine which user and group IDs correspond to Windows NT user and group RIDs.
Password Synchronization Using pam_smbpass.sopam_smbpass is a PAM module that can be used on conforming systems to
keep the smbpasswd (Samba password) database in sync with the UNIX
password file. PAM is an API supported
under some UNIX operating systems, such as Solaris, HPUX, and Linux, that provides a
generic interface to authentication mechanisms.
This module authenticates a local smbpasswd user database. If you require
support for authenticating against a remote SMB server, or if you are
concerned about the presence of SUID root binaries on your system, it is
recommended that you use pam_winbind instead.
Options recognized by this module are shown in next table.
Options recognized by pam_smbpassdebugLog more debugging info.auditLike debug, but also logs unknown usernames.use_first_passDo not prompt the user for passwords; take them from PAM_ items instead.try_first_passTry to get the password from a previous PAM module; fall back to prompting the user.use_authtokLike try_first_pass, but *fail* if the new PAM_AUTHTOK has not been previously set (intended for stacking password modules only).not_set_passDo not make passwords used by this module available to other modules.nodelaydDo not insert ~1-second delays on authentication failure.nulloknNull passwords are allowed.nonullNull passwords are not allowed. Used to override the Samba configuration.migrateoOnly meaningful in an auth context; used to update smbpasswd file with a password used for successful authentication.smbconf=fileSpecify an alternate path to the &smb.conf; file.
The following are examples of the use of pam_smbpass.so in the format of the Linux
/etc/pam.d/ files structure. Those wishing to implement this
tool on other platforms will need to adapt this appropriately.
Password Synchronization Configuration
The following is a sample PAM configuration that shows the use of pam_smbpass to make
sure private/smbpasswd is kept in sync when /etc/passwd (/etc/shadow)
is changed. It is useful when an expired password might be changed by an
application (such as ssh).
#%PAM-1.0
# password-sync
#
auth requisite pam_nologin.so
auth required pam_unix.so
account required pam_unix.so
password requisite pam_cracklib.so retry=3
password requisite pam_unix.so shadow md5 use_authtok try_first_pass
password required pam_smbpass.so nullok use_authtok try_first_pass
session required pam_unix.so
Password Migration Configuration
The following PAM configuration shows the use of pam_smbpass to migrate
from plaintext to encrypted passwords for Samba. Unlike other methods,
this can be used for users who have never connected to Samba shares:
password migration takes place when users ftp in, login using ssh, pop
their mail, and so on.
#%PAM-1.0
# password-migration
#
auth requisite pam_nologin.so
# pam_smbpass is called IF pam_unix succeeds.
auth requisite pam_unix.so
auth optional pam_smbpass.so migrate
account required pam_unix.so
password requisite pam_cracklib.so retry=3
password requisite pam_unix.so shadow md5 use_authtok try_first_pass
password optional pam_smbpass.so nullok use_authtok try_first_pass
session required pam_unix.so
Mature Password Configuration
The following is a sample PAM configuration for a mature smbpasswd installation.
private/smbpasswd is fully populated, and we consider it an error if
the SMB password does not exist or does not match the UNIX password.
#%PAM-1.0
# password-mature
#
auth requisite pam_nologin.so
auth required pam_unix.so
account required pam_unix.so
password requisite pam_cracklib.so retry=3
password requisite pam_unix.so shadow md5 use_authtok try_first_pass
password required pam_smbpass.so use_authtok use_first_pass
session required pam_unix.so
Kerberos Password Integration Configuration
The following is a sample PAM configuration that shows pam_smbpass used together with
pam_krb5. This could be useful on a Samba PDC that is also a member of
a Kerberos realm.
#%PAM-1.0
# kdc-pdc
#
auth requisite pam_nologin.so
auth requisite pam_krb5.so
auth optional pam_smbpass.so migrate
account required pam_krb5.so
password requisite pam_cracklib.so retry=3
password optional pam_smbpass.so nullok use_authtok try_first_pass
password required pam_krb5.so use_authtok try_first_pass
session required pam_krb5.so
Common Errors
PAM can be fickle and sensitive to configuration glitches. Here we look at a few cases from
the Samba mailing list.
pam_winbind Problem
A user reported, I have the following PAM configuration:
auth required /lib/security/pam_securetty.so
auth sufficient /lib/security/pam_winbind.so
auth sufficient /lib/security/pam_unix.so use_first_pass nullok
auth required /lib/security/pam_stack.so service=system-auth
auth required /lib/security/pam_nologin.so
account required /lib/security/pam_stack.so service=system-auth
account required /lib/security/pam_winbind.so
password required /lib/security/pam_stack.so service=system-auth
When I open a new console with [ctrl][alt][F1], I can't log in with my user pitie.
I have tried with user scienceu\pitie also.
The problem may lie with the inclusion of pam_stack.so
service=system-auth. That file often contains a lot of stuff that may
duplicate what you are already doing. Try commenting out the pam_stack lines
for auth and account and see if things work. If they do, look at
/etc/pam.d/system-auth and copy only what you need from it into your
/etc/pam.d/login file. Alternatively, if you want all services to use
Winbind, you can put the Winbind-specific stuff in /etc/pam.d/system-auth.
Winbind Is Not Resolving Users and Groups
My &smb.conf; file is correctly configured. I have specified
12000
and 3000-3500,
and winbind is running. When I do the following it all works fine.
&rootprompt;wbinfo -u
MIDEARTH\maryo
MIDEARTH\jackb
MIDEARTH\ameds
...
MIDEARTH\root
&rootprompt;wbinfo -g
MIDEARTH\Domain Users
MIDEARTH\Domain Admins
MIDEARTH\Domain Guests
...
MIDEARTH\Accounts
&rootprompt;getent passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/bin/bash
...
maryo:x:15000:15003:Mary Orville:/home/MIDEARTH/maryo:/bin/false
But this command fails:
&rootprompt;chown maryo a_file
chown: 'maryo': invalid user
This is driving me nuts! What can be wrong?
Your system is likely running nscd, the name service
caching daemon. Shut it down, do not restart it! You will find your problem resolved.