Rss Feed
Tweeter button
Facebook button
Linkedin button
Digg button
Youtube button

Archive for the “Linux” Category

Thursday, October 29, 2009 Categorized under Linux

Ubuntu 9.10 (Karmic Koala) The Legend

The all new OS from canonical is out. I didnt try it yet, will do it today.

A full specification is available at http://www.ubuntu.com/getubuntu/releasenotes/910overview

  • Gnome 2.28
  • Upstart (faster booting)
  • Empathy IM (replaces Pidgin)
  • New login manager
  • Quickly (easier application development)
  • Ubuntu One (cloud-based sharing and storage)
  • Linux Kernel 2.6.31
  • Changes to power management
  • New Intel video driver architecture
  • Ext4 file system default
  • Grub 2 default

So what are you waiting for. Click here to download your copy.

You could even request  your free copy here .

Monday, July 20, 2009 Categorized under Linux

Ubuntu LDAP+Samba Installation

Lightweight Directory Access Protocol:

LDAP is an acronym for Lightweight Directory Access Protocol, it is a simplified version of the X.500 protocol. The directory setup in this section will be used for authentication. Nevertheless, LDAP can be used in numerous ways: authentication, shared directory (for mail clients), address book, etc.

To describe LDAP quickly, all information is stored in a tree structure. A directory is used to store data, organized according to classes and presented in a hierarchical tree-like structure.

Directory Structure:

A directory’s structure can be compared to a file system. A directory contains entries like a folder would contain files. Each entries contain attributes inside it, with each attribute containing one or more values. Entries at the same level of a directory generally contain similar information.

A directory could contain information about people, with one entry per person. Then for each person you would have various information, also called attributes. For example a common name, surname, phone number, email address, etc. The attributes may contain one or more values. For example each person only have one common name but may have more than one phone number.

Each entry contain an attribute named objectClass that define what kind of object it is. The class determine the attributes available for the object. For example when objectClass = person the entry contain the required attributes objectClass, cn (common name) and sn (surname) and the optional attributes description.

There is three different kinds of object classes:

  • Structural Class: basic description of objects in the directory. One entry is always an instance of one structural class. For example people, group, organizational unit.
  • Auxiliary Class: add complementary information to entries. For example mailRecipient adds various attributes related to the mailbox of a person.
  • Abstract Class: represents system LDAP objects like top or alias.

Technical terms:

1, Distinguished Name:

Each entry has an attribute that is unique among all siblings of a single parent. This unique attribute is called the Relative Distinguished Name (RDN). It means that we can uniquely identify any entry within a directory by combining the RDNs of all the entries in the path from the desired node to the root of the tree. This string created by combining RDNs to form a unique name is called the node’s Distinguished Name (DN).

The DN is like the absolute path between the root of a filesystem and a file, a RDN is like a filename.

2, Base DN:

The top level of the LDAP directory tree is the base, referred to as the Base DN. A Base DN usually takes this form: dc=example, dc=com. The URL is split into domain components: example.com becomes dc=example, dc=com.

3, Directory Information Tree:

                         dc=com
                           |
                      dc=example
                      /    |    \
                     /     |     \
                    /      |      \
	      ou=People  ou=Group  ou=Computers
              /      |
             /       |
            /        |
           /         |
          /          |
         /           |
        /            |
   cn=Luke       cn=Chewbacca

Before beginning, you should determine what the root of your LDAP directory will be. By default, your tree will be determined by your Fully Qualified Domain Name (FQDN). If your domain is example.com (which we will use in this example), your root node will be dc=example,dc=com.

Installation (LDAP)

First, install the OpenLDAP server daemon slapd and ldap-utils, a package containing LDAP management utilities:

sudo apt-get install slapd ldap-utils migrationtools

The installation process will prompt you for the LDAP directory admin password and confirmation.

By default the directory suffix will match the domain name of the server. For example, if the machine’s Fully Qualified Domain Name (FQDN) is ldap.example.com, the default suffix will be dc=example,dc=com. If you require a different suffix, the directory can be reconfigured using dpkg-reconfigure. Enter the following in a terminal prompt:

sudo dpkg-reconfigure slapd

You will then be taken through a menu based configuration dialog, allowing you to configure various slapd options.

Configuration

OpenLDAP uses a separate database which contains the cn=config Directory Information Tree (DIT). The cn=config DIT is used to dynamically configure the slapd daemon, allowing the modification of schema definitions, indexes, ACLs, etc without stopping the service.

The cn=config tree can be manipulated using the utilities in the ldap-utils package. For example:

  • Use ldapsearch to view the tree, entering the admin password set during installation or reconfiguration:
    ldapsearch -xLLL -b cn=config -D cn=admin,cn=config -W olcDatabase={1}hdb
    Enter LDAP Password:
    dn: olcDatabase={1}hdb,cn=config
    objectClass: olcDatabaseConfig
    objectClass: olcHdbConfig
    olcDatabase: {1}hdb
    olcDbDirectory: /var/lib/ldap
    olcSuffix: dc=example,dc=com
    olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=exampl
     e,dc=com" write by anonymous auth by self write by * none
    olcAccess: {1}to dn.base="" by * read
    olcAccess: {2}to * by dn="cn=admin,dc=example,dc=com" write by * read
    olcLastMod: TRUE
    olcDbCheckpoint: 512 30
    olcDbConfig: {0}set_cachesize 0 2097152 0
    olcDbConfig: {1}set_lk_max_objects 1500
    olcDbConfig: {2}set_lk_max_locks 1500
    olcDbConfig: {3}set_lk_max_lockers 1500
    olcDbIndex: objectClass eq
    

    The output above is the current configuration options for the hdb backend database. Which in this case containes the dc=example,dc=com suffix.

  • Refine the search by supplying a filter, in this case only show which attributes are indexed:
    ldapsearch -xLLL -b cn=config -D cn=admin,cn=config -W olcDatabase={1}hdb olcDbIndex
    Enter LDAP Password:
    dn: olcDatabase={1}hdb,cn=config
    olcDbIndex: objectClass eq
    
  • As an example of modifying the cn=config tree, add another attribute to the index list using ldapmodify:
    ldapmodify -x -D cn=admin,cn=config -W
    Enter LDAP Password:
    dn: olcDatabase={1}hdb,cn=config
    add: olcDbIndex
    olcDbIndex: entryUUID eq
    
    modifying entry "olcDatabase={1}hdb,cn=config"
    

    Once the modification has completed, press Ctrl+D to exit the utility.

  • ldapmodify can also read the changes from a file. Copy and paste the following into a file named uid_index.ldif:
    dn: olcDatabase={1}hdb,cn=config
    add: olcDbIndex
    olcDbIndex: uid eq,pres,sub

    Then execute ldapmodify:

    ldapmodify -x -D cn=admin,cn=config -W -f uid_index.ldif
    Enter LDAP Password:
    modifying entry "olcDatabase={1}hdb,cn=config"
    

    The file method is very useful for large changes.

  • Adding additional schemas to slapd requires the schema to be converted to LDIF format. Fortunately, the slapd program can be used to automate the conversion. The following example will add the misc.schema:
    1. First, create a conversion schema_convert.conf file containing the following lines:
      include /etc/ldap/schema/core.schema
      include /etc/ldap/schema/collective.schema
      include /etc/ldap/schema/corba.schema
      include /etc/ldap/schema/cosine.schema
      include /etc/ldap/schema/duaconf.schema
      include /etc/ldap/schema/dyngroup.schema
      include /etc/ldap/schema/inetorgperson.schema
      include /etc/ldap/schema/java.schema
      include /etc/ldap/schema/misc.schema
      include /etc/ldap/schema/nis.schema
      include /etc/ldap/schema/openldap.schema
      include /etc/ldap/schema/ppolicy.schema
    2. Next, create a temporary directory to hold the output:
      mkdir /tmp/ldif_output
    3. Now using slaptest convert the schema files to LDIF:
      slaptest -f schema_convert.conf -F /tmp/ldif_output

      Adjust the configuration file name and temporary directory names if yours are different. Also, it may be worthwhile to keep the ldif_output directory around in case you want to add additional schemas in the future.

    4. Edit the /tmp/ldif_output/cn=config/cn=schema/cn={8}misc.ldif file, changing the following attributes:
      dn: cn=misc,cn=schema,cn=config
      ...
      cn: misc

      And remove the following lines from the bottom of the file:

      structuralObjectClass: olcSchemaConfig
      entryUUID: 10dae0ea-0760-102d-80d3-f9366b7f7757
      creatorsName: cn=config
      createTimestamp: 20080826021140Z
      entryCSN: 20080826021140.791425Z#000000#000#000000
      modifiersName: cn=config
      modifyTimestamp: 20080826021140Z
    5. Finally, using the ldapadd utility, add the new schema to the directory:
      ldapadd -x -D cn=admin,cn=config -W -f /tmp/ldif_output/cn\=config/cn\=schema/cn\=\{8\}misc.ldif

    There should now be a dn: cn={4}misc,cn=schema,cn=config entry in the cn=config tree.

Populating LDAP

The directory has been created during installation and reconfiguration, and now it is time to populate it. It will be populated with a “classical” scheme that will be compatible with address book applications and with Unix Posix accounts. Posix accounts will allow authentication to various applications, such as web applications, email Mail Transfer Agent (MTA) applications, etc.

cd /usr/share/migrationtools/

We need to edit the default migrationtools’ config file migrate_common.ph and
replace the following parameters with:

$DEFAULT_MAIL_DOMAIN = “example.com”;
$DEFAULT_BASE = “dc=example,dc=com”;

Then export the values:

# ./migrate_group.pl /etc/group ~/group.ldif
# ./migrate_passwd.pl /etc/passwd ~/passwd.ldif

Unfortunately, the script does not create the Group and People nodes, so we
need to create it. To do this, create a file called ~/people_group.ldif and
fill it up with:

dn: ou=people, dc=example, dc=com
ou: people
objectclass: organizationalUnit

dn: ou=group, dc=example, dc=example
ou: group
objectclass: organizationalUnit

Now, we have our users and groups converted to LDAP’s ldif format. Let import
them into our LDAP database.

# cd
# ldapadd -x -W -D “cn=admin,dc=example,dc=com” -f ~/people_group.ldif
# ldapadd -x -W -D “cn=admin,dc=example,dc=com” -f ~/group.ldif
# ldapadd -x -W -D “cn=admin,dc=example,dc=com” -f ~/passwd.ldif

where:

* -x specify that we are not using sasl
* -W prompt for password
* -D is used to identify the administrator
* -f to specify the file where ldapadd should find the data to add

Well, now the server is ready to identify your users. Let’s go on and set up
the clients.

LDAP directories can be populated with LDIF (LDAP Directory Interchange Format) files. Copy the following example LDIF file, naming it example.com.ldif, somewhere on your system:

dn: ou=people,dc=example,dc=com
objectClass: organizationalUnit
ou: people

dn: ou=groups,dc=example,dc=com
objectClass: organizationalUnit
ou: groups

dn: uid=john,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: john
sn: Doe
givenName: John
cn: John Doe
displayName: John Doe
uidNumber: 1000
gidNumber: 10000
userPassword: password
gecos: John Doe
loginShell: /bin/bash
homeDirectory: /home/john
shadowExpire: -1
shadowFlag: 0
shadowWarning: 7
shadowMin: 8
shadowMax: 999999
shadowLastChange: 10877
mail: john.doe@example.com
postalCode: 31000
l: Toulouse
o: Example
mobile: +33 (0)6 xx xx xx xx
homePhone: +33 (0)5 xx xx xx xx
title: System Administrator
postalAddress:
initials: JD

dn: cn=example,ou=groups,dc=example,dc=com
objectClass: posixGroup
cn: example
gidNumber: 10000

In this example the directory structure, a user, and a group have been setup. In other examples you might see the objectClass: top added in every entry, but that is the default behaviour so you do not have to add it explicitly.

To add the entries to the LDAP directory use the ldapadd utility:

ldapadd -x -D cn=admin,dc=example,dc=com -W -f example.com.ldif

We can check that the content has been correctly added with the tools from the ldap-utils package. In order to execute a search of the LDAP directory:

ldapsearch -xLLL -b "dc=example,dc=com" uid=john sn givenName cn

dn: uid=john,ou=people,dc=example,dc=com
cn: John Doe
sn: Doe
givenName: John

Just a quick explanation:

  • -x: will not use SASL authentication method, which is the default.
  • -LLL: disable printing LDIF schema information.

LDAP replication

LDAP often quickly becomes a highly critical service to the network. Multiple systems will come to depend on LDAP for authentication, authorization, configuration, etc. It is a good idea to setup a redundant system through replication.

Replication is achieved using the Syncrepl engine. Syncrepl allows the directory to be synced using either a push or pull based system. In a push based configuration a “primary” server will push directory updates to “secondary” servers, while a pull based approach allows replication servers to sync on a time based interval.

The following is an example of a Multi-Master configuration. In this configuration each OpenLDAP server is configured for both push and pull replication.

  1. First, configure the server to sync the cn=config database. Copy the following to a file named syncrepl_cn-config.ldif:
    dn: cn=module{0},cn=config
    changetype: modify
    add: olcModuleLoad
    olcModuleLoad: syncprov
    
    dn: cn=config
    changetype: modify
    replace: olcServerID
    olcServerID: 1 ldap://ldap01.example.com
    olcServerID: 2 ldap://ldap02.example.com
    
    dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config
    changetype: add
    objectClass: olcOverlayConfig
    objectClass: olcSyncProvConfig
    olcOverlay: syncprov
    
    dn: olcDatabase={0}config,cn=config
    changetype: modify
    add: olcSyncRepl
    olcSyncRepl: rid=001 provider=ldap://ldap01.example.com binddn="cn=admin,cn=config" bindmethod=simple
      credentials=secret searchbase="cn=config" type=refreshAndPersist
      retry="5 5 300 5" timeout=1
    olcSyncRepl: rid=002 provider=ldap://ldap02.example.com binddn="cn=admin,cn=config" bindmethod=simple
      credentials=secret searchbase="cn=config" type=refreshAndPersist
      retry="5 5 300 5" timeout=1
    -
    add: olcMirrorMode
    olcMirrorMode: TRUE
  2. Edit the file changing:
    • ldap://ldap01.example.com and ldap://ldap02.example.com to the hostnames of your LDAP servers.
    • And adjust credentials=secret to match your admin password.
  3. Next, add the LDIF file using the ldapmodify utility:
    ldapmodify -x -D cn=admin,cn=config -W -f syncrepl_cn-config.ldif
  4. Copy the syncrepl_cn-config.ldif file to the next LDAP server and repeat the ldapmodify command above.
  5. Because a new module has been added, the slapd daemon, on all replicated servers, needs to be restarted:
    sudo /etc/init.d/slapd restart
  6. Now that the configuration database is synced between servers, the backend database needs to be synced as well. Copy and paste the following into another LDIF file named syncrepl_backend.ldif:
    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    add: olcRootDN
    olcRootDN: cn=admin,dc=example,dc=com
    -
    add: olcSyncRepl
    olcSyncRepl: rid=003 provider=ldap://ldap01.example.com binddn="cn=admin,dc=example,dc=com"
     bindmethod=simple credentials=secret searchbase="dc=example,dc=com" type=refreshOnly
     interval=00:00:00:10 retry="5 5 300 5" timeout=1
    olcSyncRepl: rid=004 provider=ldap://ldap02.example.com binddn="cn=admin,dc=example,dc=com"
     bindmethod=simple credentials=secret searchbase="dc=example,dc=com" type=refreshOnly
     interval=00:00:00:10 retry="5 5 300 5" timeout=1
    -
    add: olcMirrorMode
    olcMirrorMode: TRUE
    
    dn: olcOverlay=syncprov,olcDatabase={1}hdb,cn=config
    changetype: add
    objectClass: olcOverlayConfig
    objectClass: olcSyncProvConfig
    olcOverlay: syncprov
  7. Like the previous LDIF file, edit this one changing:
    • searchbase=”dc=example,dc=com” to your directory’s searchbase.
    • If you use a different admin user, change binddn=”cn=admin,dc=example,dc=com”.
    • Also, replace credentials=secret with your admin password.
  8. Add the LDIF file:
    ldapmodify -x -D cn=admin,cn=config -W -f syncrepl_backend.ldif

    Because the servers’ configuration is already synced there is no need to copy this LDIF file to the other servers.

The configuration and backend databases should now sycnc to the other servers. You can add additional servers using the ldapmodify utility as the need arises.

In the /etc/hosts add,

127.0.0.1	ldap01.example.com ldap01

Setting up ACL

Authentication requires access to the password field, that should be not accessible by default. Also, in order for users to change their own password, using passwd or other utilities, shadowLastChange needs to be accessible once a user has authenticated.

To view the Access Control List (ACL), use the ldapsearch utility:

ldapsearch -xLLL -b cn=config -D cn=admin,cn=config -W olcDatabase=hdb olcAccess
Enter LDAP Password:
dn: olcDatabase={1}hdb,cn=config
olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=exampl
 e,dc=com" write by anonymous auth by self write by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by dn="cn=admin,dc=example,dc=com" write by * read

TLS and SSL

When authenticating to an OpenLDAP server it is best to do so using an encrypted session. This can be accomplished using Transport Layer Security (TLS) and/or Secure Sockets Layer (SSL).

Once you have a certificate, key, and CA cert installed, use ldapmodify to add the new configuration options:

ldapmodify -x -D cn=admin,cn=config -W
Enter LDAP Password:
dn: cn=config
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/cacert.pem
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ssl/certs/server.crt
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ssl/private/server.key

modifying entry "cn=config"

Next, edit /etc/default/slapd uncomment the SLAPD_SERVICES option:

SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///"

Now the openldap user needs access to the certificate:

sudo adduser openldap ssl-cert
sudo chgrp ssl-cert /etc/ssl/private/server.key

Finally, restart slapd:

sudo /etc/init.d/slapd restart

The slapd daemon should now be listening for LDAPS connections and be able to use STARTTLS during authentication.

TLS Replication

If you have setup Syncrepl between servers, it is prudent to encrypt the replication traffic using Transport Layer Security (TLS). For details on setting up replication see the section called “LDAP replication”.

After setting up replication, and following the instructions in the section called “TLS and SSL”, there are a couple of consequences that should be kept in mind:

  • The configuration only needs to be modified on one server.
  • The path names for the certificate and key must be the same on all servers.

So on each replicated server: install a certificate, edit /etc/default/slapd, and restart slapd.

Once TLS has been setup on each server, modify the cn=config replication by entering the following in a terminal:

ldapmodify -x -D cn=admin,cn=config -W
Enter LDAP Password:
dn: olcDatabase={0}config,cn=config
replace: olcSyncrepl
olcSyncrepl: {0}rid=001 provider=ldap://ldap01.example.com binddn="cn=admin,cn
 =config" bindmethod=simple credentials=secret searchbase="cn=config" type=refre
 shAndPersist retry="5 5 300 5" timeout=1 starttls=yes
olcSyncrepl: {1}rid=002 provider=ldap://ldap02.example.com binddn="cn=admin,cn
 =config" bindmethod=simple credentials=secret searchbase="cn=config" type=refre
 shAndPersist retry="5 5 300 5" timeout=1 starttls=yes

modifying entry "olcDatabase={0}config,cn=config"

Now adjust the backend database replication:

ldapmodify -x -D cn=admin,cn=config -W
Enter LDAP Password:
dn: olcDatabase={1}hdb,cn=config
replace: olcSyncrepl
olcSyncrepl: {0}rid=003 provider=ldap://ldap01.example.com binddn="cn=admin,dc=example,dc=
 com" bindmethod=simple credentials=secret searchbase="dc=example,dc=com" type=r
 efreshOnly interval=00:00:00:10 retry="5 5 300 5" timeout=1 starttls=yes
olcSyncrepl: {1}rid=004 provider=ldap://ldap02.example.com binddn="cn=admin,dc=example,dc=
 com" bindmethod=simple credentials=secret searchbase="dc=example,dc=com" type=r
 efreshOnly interval=00:00:00:10 retry="5 5 300 5" timeout=1 starttls=yes

modifying entry "olcDatabase={1}hdb,cn=config"

If the LDAP server hostname does not match the Fully Qualified Domain Name (FQDN) in the certificate, you may have to edit /etc/ldap/ldap.conf and add the following TLS options:

TLS_CERT /etc/ssl/certs/server.crt
TLS_KEY /etc/ssl/private/server.key
TLS_CACERT /etc/ssl/certs/cacert.pem

Finally, restart slapd on each of the servers:

sudo /etc/init.d/slapd restart

LDAP Authentication

Once you have a working LDAP server, the auth-client-config and libnss-ldap packages take the pain out of configuring an Ubuntu client to authenticate using LDAP. To install the packages from, a terminal prompt enter:

sudo apt-get install libnss-ldap

During the install a menu dialog will ask you connection details about your LDAP server.

If you make a mistake when entering your information you can execute the dialog again using:

sudo dpkg-reconfigure ldap-auth-config

The results of the dialog can be seen in /etc/ldap.conf. If your server requires options not covered in the menu edit this file accordingly.

Now that libnss-ldap is configured enable the auth-client-config LDAP profile by entering:

sudo auth-client-config -t nss -p lac_ldap
  • -t: only modifies /etc/nsswitch.conf.
  • -p: name of the profile to enable, disable, etc.
  • lac_ldap: the auth-client-config profile that is part of the ldap-auth-config package.

Using the pam-auth-update utility, configure the system to use LDAP for authentication:

sudo pam-auth-update

From the pam-auth-update menu, choose LDAP and any other authentication mechanisms you need.

You should now be able to login using user credentials stored in the LDAP directory.

User and Group Management

The ldap-utils package comes with multiple utilities to manage the directory, but the long string of options needed, can make them a burden to use. The ldapscripts package contains configurable scripts to easily manage LDAP users and groups.

To install the package, from a terminal enter:

sudo apt-get install ldapscripts

Next, edit the config file /etc/ldapscripts/ldapscripts.conf uncommenting and changing the following to match your environment:

SERVER=localhost
BINDDN='cn=admin,dc=example,dc=com'
BINDPWDFILE="/etc/ldapscripts/ldapscripts.passwd"
SUFFIX='dc=example,dc=com'
GSUFFIX='ou=Groups'
USUFFIX='ou=People'
MSUFFIX='ou=Computers'
GIDSTART=10000
UIDSTART=10000
MIDSTART=10000

Now, create the ldapscripts.passwd file to allow authenticated access to the directory:

sudo sh -c "echo -n 'secret' > /etc/ldapscripts/ldapscripts.passwd"
sudo chmod 400 /etc/ldapscripts/ldapscripts.passwd

The ldapscripts are now ready to help manage your directory. The following are some examples of how to use the scripts:

  • Create a new user:
    sudo ldapadduser george example

    This will create a user with uid george and set the user’s primary group (gid) to example

  • Change a user’s password:
    sudo ldapsetpasswd george
    Changing password for user uid=george,ou=People,dc=example,dc=com
    New Password: 
    New Password (verify): 
  • Delete a user:
    sudo ldapdeleteuser george
  • Add a group:
    sudo ldapaddgroup qa
  • Delete a group:
    sudo ldapdeletegroup qa
  • Add a user to a group:
    sudo ldapaddusertogroup george qa

    You should now see a memberUid attribute for the qa group with a value of george.

  • Remove a user from a group:
    sudo ldapdeleteuserfromgroup george qa

    The memberUid attribute should now be removed from the qa group.

  • The ldapmodifyuser script allows you to add, remove, or replace a user’s attributes. The script uses the same syntax as the ldapmodify utility. For example:
    sudo ldapmodifyuser george
    # About to modify the following entry :
    dn: uid=george,ou=People,dc=example,dc=com
    objectClass: account
    objectClass: posixAccount
    cn: george
    uid: george
    uidNumber: 1001
    gidNumber: 1001
    homeDirectory: /home/george
    loginShell: /bin/bash
    gecos: george
    description: User account
    userPassword:: e1NTSEF9eXFsTFcyWlhwWkF1eGUybVdFWHZKRzJVMjFTSG9vcHk=
    
    # Enter your modifications here, end with CTRL-D.
    dn: uid=george,ou=People,dc=example,dc=com
    replace: gecos
    gecos: George Carlin

    The user’s gecos should now be “George Carlin”.

  • Another great feature of ldapscripts, is the template system. Templates allow you to customize the attributes of user, group, and machine objectes. For example, to enable the user template edit /etc/ldapscripts/ldapscripts.conf changing:
    UTEMPLATE="/etc/ldapscripts/ldapadduser.template"

    There are sample templates in the /etc/ldapscripts directory. Copy or rename the ldapadduser.template.sample file to /etc/ldapscripts/ldapadduser.template:

    sudo cp /etc/ldapscripts/ldapadduser.template.sample /etc/ldapscripts/ldapadduser.template

    Edit the new template to add the desired attributes. The following will create new user’s as with an objectClass of inetOrgPerson:

    dn: uid=<user>,<usuffix>,<suffix>
    objectClass: inetOrgPerson
    objectClass: posixAccount
    cn: <user>
    sn: <ask>
    uid: <user>
    uidNumber: <uid>
    gidNumber: <gid>
    homeDirectory: <home>
    loginShell: <shell>
    gecos: <user>
    description: User account
    title: Employee

    Notice the <ask> option used for the cn value. Using <ask> will configure ldapadduser to prompt you for the attribute value during user creation.

There are more useful scripts in the package, to see a full list enter: dpkg -L ldapscripts | grep bin

Installation (+samba)

There are three packages needed when integrating Samba with LDAP. samba, samba-doc, and smbldap-tools packages . To install the packages, from a terminal enter:

sudo apt-get install samba samba-doc smbldap-tools

Strictly speaking the smbldap-tools package isn’t needed, but unless you have another package or custom scripts, a method of managing users, groups, and computer accounts is needed.

OpenLDAP Configuration

In order for Samba to use OpenLDAP as a passdb backend, the user objects in the directory will need additional attributes. This section assumes you want Samba to be configured as a Windows NT domain controller, and will add the necessary LDAP objects and attributes.

  • The Samba attributes are defined in the samba.schema file which is part of the samba-doc package. The schema file needs to be unzipped and copied to /etc/ldap/schema. From a terminal prompt enter:
    sudo cp /usr/share/doc/samba-doc/examples/LDAP/samba.schema.gz /etc/ldap/schema/
    sudo gzip -d /etc/ldap/schema/samba.schema.gz
  • The samba schema needs to be added to the cn=config tree. The procedure to add a new schema to slapd .
    1. First, create a configuration file named schema_convert.conf, or a similar descriptive name, containing the following lines:
      include /etc/ldap/schema/core.schema
      include /etc/ldap/schema/collective.schema
      include /etc/ldap/schema/corba.schema
      include /etc/ldap/schema/cosine.schema
      include /etc/ldap/schema/duaconf.schema
      include /etc/ldap/schema/dyngroup.schema
      include /etc/ldap/schema/inetorgperson.schema
      include /etc/ldap/schema/java.schema
      include /etc/ldap/schema/misc.schema
      include /etc/ldap/schema/nis.schema
      include /etc/ldap/schema/openldap.schema
      include /etc/ldap/schema/ppolicy.schema
      include /etc/ldap/schema/samba.schema
    2. Next, create a temporary directory to hold the output:
      mkdir /tmp/ldif_output
    3. Now use slaptest to convert the schema files:
      slaptest -f schema_convert.conf -F /tmp/ldif_output

      Change the above file and path names to match your own if they are different.

    4. Edit the generated /tmp/ldif_output/cn=config/cn=schema/cn={12}samba.ldif file, changing the following attributes:
      dn: cn=samba,cn=schema,cn=config
      ...
      cn: samba

      And remove the following lines from the bottom of the file:

      structuralObjectClass: olcSchemaConfig
      entryUUID: b53b75ca-083f-102d-9fff-2f64fd123c95
      creatorsName: cn=config
      createTimestamp: 20080827045234Z
      entryCSN: 20080827045234.341425Z#000000#000#000000
      modifiersName: cn=config
      modifyTimestamp: 20080827045234Z
    5. Finally, using the ldapadd utility, add the new schema to the directory:
      ldapadd -x -D cn=admin,cn=config -W -f /tmp/ldif_output/cn\=config/cn\=schema/cn\=\{12\}samba.ldif

    There should now be a dn: cn={X}misc,cn=schema,cn=config, where “X” is the next sequential schema, entry in the cn=config tree.

  • Copy and paste the following into a file named samba_indexes.ldif:
    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    add: olcDbIndex
    olcDbIndex: uidNumber eq
    olcDbIndex: gidNumber eq
    olcDbIndex: loginShell eq
    olcDbIndex: uid eq,pres,sub
    olcDbIndex: memberUid eq,pres,sub
    olcDbIndex: uniqueMember eq,pres
    olcDbIndex: sambaSID eq
    olcDbIndex: sambaPrimaryGroupSID eq
    olcDbIndex: sambaGroupType eq
    olcDbIndex: sambaSIDList eq
    olcDbIndex: sambaDomainName eq
    olcDbIndex: default sub

    Using the ldapmodify utility load the new indexes:

    ldapmodify -x -D cn=admin,cn=config -W -f samba_indexes.ldif

    If all went well you should see the new indexes using ldapsearch:

    ldapsearch -xLLL -D cn=admin,cn=config -x -b cn=config -W olcDatabase={1}hdb
  • Next, configure the smbldap-tools package to match your environment. The package comes with a configuration script that will ask questions about the needed options. To run the script enter:
    sudo gzip -d /usr/share/doc/smbldap-tools/configure.pl.gz
    sudo perl /usr/share/doc/smbldap-tools/configure.pl

    Once you have answered the questions, there should be /etc/smbldap-tools/smbldap.conf and /etc/smbldap-tools/smbldap_bind.conf files. These files are generated by the configure script, so if you made any mistakes while executing the script it may be simpler to edit the file appropriately.

  • The smbldap-populate script will add the necessary users, groups, and LDAP objects required for Samba. It is a good idea to make a backup LDAP Data Interchange Format (LDIF) file with slapcat before executing the command:
    sudo slapcat -l backup.ldif
  • Once you have a current backup execute smbldap-populate by entering:
    sudo smbldap-populate

Your LDAP directory now has the necessary domain information to authenticate Samba users.

Samba Configuration

There a multiple ways to configure Samba for details on some common configurations see Chapter 15, Windows Networking. To configure Samba to use LDAP, edit the main Samba configuration file /etc/samba/smb.conf commenting the passdb backend option and adding the following:

#   passdb backend = tdbsam

# LDAP Settings
   passdb backend = ldapsam:ldap://hostname
   ldap suffix = dc=example,dc=com
   ldap user suffix = ou=People
   ldap group suffix = ou=Groups
   ldap machine suffix = ou=Computers
   ldap idmap suffix = ou=Idmap
   ldap admin dn = cn=admin,dc=example,dc=com
   ldap ssl = start tls
   ldap passwd sync = yes
...
   add machine script = sudo /usr/sbin/smbldap-useradd -t 0 -w "%u"

Restart samba to enable the new settings:

sudo /etc/init.d/samba restart

Now Samba needs to know the LDAP admin password. From a terminal prompt enter:

sudo smbpasswd -w secret

If you currently have users in LDAP, and you want them to authenticate using Samba, they will need some Samba attributes defined in the samba.schema file. Add the Samba attributes to existing users using the smbpasswd utility, replacing username with an actual user:

sudo smbpasswd -a username

You will then be asked to enter the user’s password.

To add new user, group, and machine accounts use the utilities from the smbldap-tools package. Here are some examples:

  • To add a new user to LDAP with Samba attributes enter the following, replacing username with an actual username:
    sudo smbldap-useradd -a -P username

    The -a option adds the Samba attributes, and the -P options calls the smbldap-passwd utility after the user is created allowing you to enter a password for the user.

  • To remove a user from the directory enter:
    sudo smbldap-userdel username

    The smbldap-userdel utility also has a -r option to remove the user’s home directory.

  • Use smbldap-groupadd to add a group, replacing groupname with an appropriate group:
    sudo smbldap-groupadd -a groupname

    Similar to smbldap-useradd, the -a adds the Samba attributes.

  • To add a user to a group use smbldap-groupmod:
    sudo smbldap-groupmod -m username groupname

    Be sure to replace username with a real user. Also, the -m option can add more than one user at a time by listing them in comma separated format.

  • smbldap-groupmod can also be used to remove a user from a group:
    sudo smbldap-groupmod -x username groupname
  • Additionally, the smbldap-useradd utility can add Samba machine accounts:
    sudo smbldap-useradd -t 0 -w username

    Replace username with the name of the workstation. The -t 0 option creates the machine account without a delay, while the -w option specifies the user as a machine account. Also, note the add machine script option in /etc/samba/smb.conf was changed to use smbldap-useradd.

Sunday, July 19, 2009 Categorized under Linux

The Ext4 filesystem

The ext4 or fourth extended filesystem is a journaling file system developed as the successor to ext3. It was born as a series of backward compatible extensions to add 64-bit storage limits and other performance improvements to ext3. However, other Linux kernel developers opposed accepting extensions to ext3 for stability reasons, and proposed to fork the source code of ext3, rename it as ext4, and do all the development there, without affecting the current ext3 users. This proposal was accepted, and on June 28, 2006 Theodore Ts’o, the ext3 maintainer, announced the new plan of development for ext4. A preliminary development snapshot of ext4 was included in version 2.6.19 of the Linux kernel. On Oct 11, 2008, the patches that mark ext4 as stable code were merged in the Linux 2.6.28 source code repositories, denoting the end of the development phase and recommending ext4 adoption. Kernel 2.6.28, containing the ext4 filesystem, was finally released on December 25, 2008.

Large file system

The ext4 filesystem can support volumes with sizes up to 1 exabyte and files with sizes up to 16 terabytes.

Extents

Extents are introduced to replace the traditional block mapping scheme used by ext2/3 filesystems. An extent is a range of contiguous physical blocks, improving large file performance and reducing fragmentation. A single extent in ext4 can map up to 128MB of contiguous space with a 4KB block size.[1] There can be 4 extents stored in the Inode. When there are more than 4 extents to a file, the rest of the extents are indexed in an Htree.

Backward compatibility

The ext4 filesystem is backward compatible with ext3 and ext2, making it possible to mount ext3 and ext2 filesystems as ext4. This will already slightly improve performance, because certain new features of ext4 can also be used with ext3 and ext2, such as the new block allocation algorithm.

The ext3 file system is partially forward compatible with ext4, that is, an ext4 filesystem can be mounted as an ext3 partition (using “ext3″ as the filesystem type when mounting). However, if the ext4 partition uses extents (a major new feature of ext4), then the ability to mount the file system as ext3 is lost.

Persistent pre-allocation

The ext4 filesystem allows for pre-allocation of on-disk space for a file. The current methodology for this on most file systems is to write the file full of 0s to reserve the space when the file is created (although XFS has an ioctl to allow for true pre-allocation as well). This method would no longer be required for ext4; instead, a new fallocate() system call was added to the linux kernel for use by filesystems, including ext4 and XFS, that have this capability. The space allocated for files such as these would be guaranteed and would likely be contiguous. This has applications for media streaming and databases.

Delayed allocation

Ext4 uses a filesystem performance technique called allocate-on-flush, also known as delayed allocation. It consists of delaying block allocation until the data is going to be written to the disk, unlike other file systems, which allocate the necessary blocks before that step. This improves performance and reduces fragmentation by improving block allocation decisions based on the actual file size.

Break 32,000 subdirectory limit

In ext3 the number of subdirectories that a directory can contain is limited to 32,000. This limit has been raised to 64,000 in ext4, and with the “dir_nlink” feature it can go beyond this (although it will stop increasing the link count on the parent). To allow for continued performance given the possibility of much larger directories, Htree indexes (a specialized version of a B-tree) are turned on by default in ext4. This feature is implemented in Linux kernel 2.6.23. Htree is also available in ext3 when the dir_index feature is enabled.

Journal checksumming

Ext4 uses checksums in the journal to improve reliability, since the journal is one of the most used files of the disk. This feature has a side benefit; it can safely avoid a disk I/O wait during the journaling process, improving performance slightly. The technique of journal checksumming was inspired by a research paper from the University of Wisconsin titled IRON File Systems (specifically, section 6, called “transaction checksums”).

Online defragmentation

There are a number of proposals for an online defragmenter, but that support is not yet included in the mainline kernel. Even with the various techniques used to avoid fragmentation, a long lived file system does tend to become fragmented over time. Ext4 will have a tool which can defragment individual files or entire file systems.

Faster file system checking

In ext4, unallocated block groups and sections of the inode table are marked as such. This enables e2fsck to skip them entirely on a check and greatly reduce the time it takes to check a file system of the size ext4 is built to support. This feature is implemented in version 2.6.24 of the Linux kernel.

Multiblock allocator

Ext4 allocates multiple blocks for a file in a single operation, which reduces fragmentation by attempting to choose contiguous blocks on the disk. The multiblock allocator is active when using O_DIRECT or if delayed allocation is on. This allows the file to have many dirty blocks submitted for writes at the same time, unlike the existing kernel mechanism of submitting each block to the filesystem separately for allocation.

Improved timestamps

As computers become faster in general and as Linux becomes used more for mission critical applications, the granularity of second-based timestamps becomes insufficient. To solve this, ext4 provides timestamps measured in nanoseconds. In addition, 2 bits of the expanded timestamp field are added to the most significant bits of the seconds field of the timestamps to defer the year 2038 problem for an additional 500 years.

Ext4 also adds support for date-created timestamps. But, as Theodore Ts’o points out, while it is easy to add an extra creation-date field in the inode (thus technically enabling support for date-created timestamps in ext4), it is more difficult to modify or add the necessary system calls, like stat() (which would probably require a new version), and the various libraries that depend on them (like glibc). These changes would require coordination of many projects. So, even if ext4 developers implement initial support for creation-date timestamps, this feature will not be available to user programs for now.[

Caveats

Delayed allocation and potential data loss

Delayed allocation poses some additional risk of data loss in cases where the system crashes before all of the data has been written to the disk.

The typical scenario in which this might occur is a program replacing the contents of a file without forcing a write to the disk with fsync. Problems can arise if the system crashes before the actual write occurs. In this situation, users of ext3 have come to expect that the disk will hold either the old version or the new version of the file following the crash. However, the ext4 code in the Linux kernel version 2.6.28 will often clear the contents of the file before the crash, but never write the new version, thus losing the contents of the file entirely.

Many people find such a behavior unacceptable. A significant issue is that fixing the bug by using fsync more often could lead to severe performance penalties on ext3 filesystems mounted with the data=ordered flag (the default on most Linux distributions). Given that both file-systems will be in use for some time, this complicates matters enormously for end-user application developers. In response, Theodore Ts’o has written some patches for ext4 that cause it to limit its delayed allocation in these common cases. For a small cost in performance, this will significantly increase the chance that either version of the file will survive the crash.

The new patches are expected to become part of the mainline kernel 2.6.30. Various distributions may choose to backport them to 2.6.28 or 2.6.29, for instance Ubuntu made them part of the 2.6.28 kernel in version 9.04—Jaunty Jackalope.