Advanced Module Development

This chapter carries on from chapter 55, and explains some of the more advanced parts of module development such as access control, logging and integration with the Users and Groups module.

Module Access Control

Webmin versions 0.72 and above support a standard method for restricting which features of a module a user can access. For example, the Apache module allows a Webmin user to be restricted to managing selected virtual servers, and the BIND module allows user to be limited to editing records only in certain domains.

This kind of detailed access control is separate from the first level ACLs that control which users have access to which modules. As long as your module calls init_config, the Webmin API will automatically block users who do not have access to the entire module.

Module access control options are set in the Webmin Users module by clicking on the name of a module next to a user's name. The options available are generated by code from the module itself (except for the Can edit module configuration? option, which is always present). When the user clicks on Save the form parameters are also parsed by code from the module being configured, before being saved in the Webmin configuration directory.

A module wanting to use access control must contain a file called acl_security.pl in its directory. This file must contain two Perl functions :

acl_security_form(acl) This function takes a reference to a hash containing the current ACL options for this user, and must output HTML for form inputs to edit those ACL options. Because the HTML will be inside a 4-column table, you must generate the appropriate and tags around your input elements.

acl_security_save(acl, inputs) This function must fill in the given hash reference with values from the form created by acl_security_form. Form inputs are available in the global hash %in as generated by ReadParse? , or from the second parameter to the function.

Because these functions are called in the context of your module, the acl_security.pl file can require the common functions file used by other CGI programs in the module. This gives you access to all the standard Webmin functions, and allows you to provide more meaningful inputs. For example, when setting ACL options for the Apache module a list of virtual servers from the Apache configuration is displayed for the user to select from.

An example acl_security.pl might look like :

do 'foo-lib.pl';

sub acl_security_form

{

print " Can edit users?\n";

printf " Yes\n",

$_[0]->{'edit'} ? 'checked' : '';

printf " No \n",

$_[0]->{'edit'} ? '' : 'checked';

}

sub acl_security_save

{

$_[0]->{'edit'} = $in{'edit'};

}

If a user has not yet had any ACL options set for a module, a default set of options will be used. These are read from the file defaultacl in the module directory, which must contain name_=_value pairs one per line. These options should allow the user to do anything, so that the admin or master Webmin user is not restricted by default.

To actually enforced the chosen ACL options for each user, your module programs must use the get_module_acl function to get the ACL for the current user, and then verify that each action is allowed. When called with no parameters this function will return a hash containing the options set for the current user in the current module, which is almost always what you want. For example :

#!/usr/local/bin/perl

require 'foo-lib.pl';

%access = &get_module_acl();

$access{'create'} ||

&error("You are not allowed to create new foo users");

When designing a module that some users will have limited access to, remember the user can enter any URL, not just those that you link to. For example, just doing ACL checking in the program that displays a form is not enough - the program that processing the form should do all the same checks as well. Similarly, CGI parameters should never be trusted, even hidden parameters that cannot normally be input by the user.

User Update Notification

Since version 0.72 it has been possible to have the Users and Groups module notify other modules when a Unix user is added, updated or deleted. This can be useful if your module deals with additional information that is associated with users. For example, the Disk Quotas module sets default quotas when new users are created, and the Samba Windows File Sharing module keeps the Samba password file in sync with the Unix user list.

To have your module notified when a user is added, updated or deleted you must create a Perl script called useradmin_update.pl in your module directory. This file must contain three functions :

useradmin_create_user(user) This function is called when a new Unix user is created. The user parameter is a hash containing the details of the new user, described in more detail below.

useradmin_modify_user(user, olduser) This function is called when an existing Unix user is modified in any way. The user parameter is a hash containing the new details of the user, and olduser the details of the user before he was modified.

useradmin_delete_user(user) This function is called when a Unix user is deleted. Like the other functions, the user hash contains the user's details.

The hash reference passed to each of the three functions has the following keys :

If the system has shadow passwords enabled, other keys may also be available - but it is not a good idea to rely on them.

When your functions are called, they will be in the context of your module. This means that your useradmin_update.pl script can require the file of common functions used by other CGI programs. The functions can perform any action you like in order to update other configuration files or whatever, but should not generate any output on STDOUT, or take too long to execute. A partial example useradmin_update.pl might look like :

do 'foo-lib.pl';

sub useradmin_create_user

{

local $lref = &read_file_lines($users_file);

push(@$lref, "$_[0]->{'user'}:$_[0]->{'pass'}");

&flush_file_lines();

}

Internationalisation

Webmin versions 0.75 and above provide module writers with functions for generating different text and messages depending on the language selected by the user. Each module that wishes to use this feature should have a subdirectory called lang which contains a translation file for each language supported. Each line of a translation file defines a message in that language in the format message_code_=_Message in this language

The default language for Webmin is English (code en), so every module should have at least a file called lang/en. If any other language is missing a message, the English one will be used instead. Check the file lang_list.txt for all the languages currently supported and their codes. To change the current language, go into the Webmin Configuration module and click on the Language icon.

When your module calls the init_config function, all the messages from the appropriate translation file will be read into the hash %text. Thus instead of generating hard-coded text like this :

print "Click here to start the server :

\n";

Your module should use the %text hash like so :

print $text{'startmsg'},"

\n";

Messages from the appropriate file in the top-level lang directory are also included in %text. Several useful messages such as save, delete and create are thus available to every module.

In some cases, you may want to include some variable text in a message. Because the position of the variable may differ depending on the language used, message strings can include place-markers like $1, $2 or $3. The function text should be used to replace these place-markers with actual values like so :

print &text('servercount', $count),"

\n";

Your module's module.info file can also support multiple languages by adding a line like desc__code_=_module description_ for each language, where code is the language code. You can also have a separate config.info file for each language, called config.info._code_, and separate help files for each language, named like intro._code_.html. In all cases, if there is no translation for the user's chosen language then the default (English) will be used instead.

File Locking

Webmin version 0.81 introduced several new common functions for locking files to prevent multiple programs from writing to them at the same time. Module programmers should make use of these functions in order to prevent the corruption or overwriting of configuration files in cases where two users are using the same module at the same time.

Locking is done by the function lock_file, which takes the name of a file as a parameter and obtains and exclusive lock on that file by creating a file with the same name but with .lock appended. Similarly, the function unlock_file removes the lock on the file given as a parameter. Because the .lock file stores the PID of the process that locked the file, any locks a CGI program holds will be automatically released when it exits. However, it is recommended that locks be properly released by calling unlock_file or unlock_all_files before exiting.

The following code shows how the locking functions might be used :

&lock_file("/etc/something.conf");

open(CONF, ">>/etc/something.conf");

print CONF "some new directive\n";

close(CONF);

&unlock_file("/etc/something.conf");

Locking should be done as soon as possible in the CGI program, ideally before reading the file to be changed and definitely before writing to it. Files can and should be locked during creation and deletion as well, as should directories and symbolic links before creation or removal. While this is not really necessary to prevent file corruption, it does make the logging of file changes performed by the program more complete, as explained below.

Many other programs also use .lock files for the same purpose, but most do not put their process ID in the file. If the lock_file function encounters a lock like this, it will wait until it is completely removed before obtaining its own lock, as there is no way to tell if the original process is still running or not.

Action Logging

Webmin versions 0.81 and above have support for detailed logging by CGI programs of the actions performed by users for later viewing in the Webmin Actions Log module. While previous versions wrote a HTTP log file to /var/webmin/miniserv.log, this did not contain the information required to work out exactly what each Webmin user had been doing. To improve on this, Webmin now logs detailed information to the file /var/webmin/webmin.log and optionally to files in the directory /var/webmin/diffs. Note that nothing will be recorded in this file if logging is not enabled in the Webmin Configuration module.

The function webmin_log should be called by CGI programs after they have successfully completed all processing and file updates. The parameters taken by the function are :

action The action the program has performed. Usually something like save or delete.

type The type of thing effected by the program. Often something like user or group, though can be left blank if not appropriate.

object The name of the thing effected, such as jcameron or root or www.foobar.com.

parameters A reference to a hash containing additional information that the program wants to log. Often just passing \%in is useful.

All of these parameters can contain any information you want, as they are merely logged to the actions log file and not interpreted by webmin_log in any way. For example, a module might call the function like this :

&lock_file("/etc/foo.users");

open(USERS, ">>/etc/foo.users");

print USERS "$in{'username'} $in{'password'}\n";

close(USERS);

&unlock_file("/etc/foo.users");

&webmin_log("create", "user", $in{'username'}, \%in);

Because the raw log files are not easy to understand, Webmin also provides support for converting detailed action logs into human-readable format. The Webmin Actions Log module makes use of a Perl function in the file log_parser.pl in each module's subdirectory to convert logs records from that module into a readable message.

This file must contain the function parse_webmin_log, which is called once for each log record for this module. It will be called with the following parameters :

user The Webmin user who run the program that generated this log record.

script The filename of the CGI script that generated this log, without the directory.

action Whatever was passed as the action parameter to webmin_log to create this log record.

type Whatever was passed as the type parameter to webmin_log.

object Whatever was passed as the object parameter to webmin_log.

parameters A reference to a hash the same as the one passed to webmin_log.

long If non-zero, this indicates that the function is being called to create the description for the Action Details page, and thus can return a longer message than normal. You can ignore this if you like.

The function should return a text string based on the parameters passed to it that converts them into a readable description for the user. For example, your log_parser.pl file might look like :

require 'foo-lib.pl';

sub parse_webmin_log

{

local ($user, $script, $action, $type, $object, $params, $long) = @_;

if ($action eq 'create') {

return &text('log_create', $user);

}

elsif ($action eq 'delete') {

return &text('log_delete', $user);

}

}

Because the log_parser.pl file is read and executed in a similar way to how the acl_security.pl file is handled by the Webmin Users module, it can require the module's own library of functions just like any module CGI program would. This means that the text function and %text hash are available for accessing the module's translated text strings, as in the example above.

Webmin can also be configured to record exactly what file changes have been made by each CGI program before calling webmin_log. Under Logging in the Webmin Configuration module is a checkbox labeled Log changes made to files by each action which when enabled will cause the webmin_log function to use the diff command to find changes made to any file locked by each program.

When logging of file changes is enabled, the Action Details page in the actions log module will show the diffs for all files updates, creations and deletions by the chosen action. If locking of directories and symbolic links is done as well, it will show their creations and modifications too.

As well as having their file changes logged, programs can also use the common functions system_logged, kill_logged and rename_logged which take the same parameters as the Perl system, kill and rename functions, but also record the event for viewing on the *Action Details* page. There is also a backquote_logged function which works similar to the Perl backquote operator (it takes a command and executes it, returning the output), but also logs the command. If these functions are used they must be called before webmin_log for the logging to be actually recorded, as in this example :

if ($pid) {

&kill_logged('TERM', $pid);

}

else {

&system_logged("/etc/init.d/foo stop");

}

&webmin_log("stop");

Pre and Post Install Scripts

Webmin versions 0.990 and above allow modules to contain Perl scripts that will be run after a module is installed and before it is uninstalled. If your module contains a file called postinstall.pl, the Perl function module_install in this file will be called after the install of your module is complete. Because it is executed in the module's directory, it can make use of the common functions library, like so :

require 'yourmodule-lib.pl';

sub module_install

{

system("cp $module_root_directory/somefile $config_directory/somefile")

if (!-r "$config_directory/somefile");

}

The function will be called when a module is installed from the Webmin Configuration or Cluster Webmin Servers modules. However, it is not called if the install-module.pl script is used, or when the module in RPM format is installed.

Similarly, if your module contains a file called uninstall.pl, the Perl function module_uninstall in that file will be called just before the module is deleted. This can happen when it is deleted using the Webmin Users or Cluster Webmin Servers modules, or when the entire of Webmin is uninstalled. The uninstall function should clean up any configuration that will no longer work when the module is uninstalled, such as Cron jobs that reference scripts in the module.

Functions in Other Modules

The standard Webmin modules contain a vast number of useful functions for parsing and manipulating the configuration files for Apache, BIND, Unix Users and so on. If your module needs to configure these servers as well in some way, it makes sense to make use of existing functions in the standard modules.

Because the standard modules have typically already been configured with the correct paths for files like httpd.conf and squid.conf, their functions will use those paths when you call them to read and write configuration files. The actual %config settings for another module can also be accessed, so that your module knows what commands to use to apply changes to or start some server like Apache or Squid.

When you first load the library for some other module with the foreign_require function, it is actually executed in a separate Perl module namespace. All of your module's CGI programs and its library will be in the main namespace, but other foreign module's functions will be put in a namespace with the same name as the Webmin module. This means that you can call those functions with code like &useradmin::list_users(), and access global variables like $useradmin::config{'passwd_file'}. This Perl namespace separation ensures that functions and globals with the same names can exist in both your and the foreign module, without any clashes. Some things are shared between all modules though, such as caches used by get_system_hostname, load_language, read_file_cached and get_all_module_infos, so that loading the library of a new module with foreign_require is not too slow.

The only way to find out which functions are available in other modules is to read the source code of their CGI programs and libraries. Unfortunately there is no documentation at the moment as to what the functions do, apart from maybe some comments in the source code. You will also find that not every feature of another module is accessible though its functions - for example, the Apache module does not have a function for applying the current configuration. Instead, this is done by the restart.cgi script, which runs the appropriate commands directly.

Probably the most useful module for others to make use of is Running Processes, as it contains several functions for starting processes in the background. Many of the standard modules make use of them, which is why Running Processes should never be un-installed unless you want to break several other Webmin modules as well.

The functions available after calling &foreign_require("proc", "proc-lib.pl"); are :

proc::safe_process_exec(command, uid, gid, handle, input, fixtags, bsmode) Executes the specified command as the given Unix uid and gid, and writes its output to a Perl file handle (usually STDOUT). This can be useful when executing programs that fork their own sub-processes, which would normally prevent Perl from detecting the end of their output when run using a piped open function call. For example, many servers have startup scripts like /etc/init.d/httpd that exhibit this problematic behaviour. If the input parameter is supplied, it will be fed as input to the command when run. If the fixtags parameter is set to 1, all output from the command will have HTML characters escaped that that it can be properly displayed in a browser. And if the bsmode argument is set, any backspace or return characters output by the command are interpreted to remove the last output letter or clear the current line, respectively. This can be useful if the program usually uses these characters to display an incrementing counter, as mkfs and cdrecord do.

proc::safe_process_exec_logged(command, uid, gid, handle, input, fixtags, bsmode) This function behaves just like safe_process_exec, but also records the executed command so that it will be logged when webmin_log is called.

proc::pty_process_exec(command, [uid, gid]) Executes the specified command in a new PTY as either the given Unix uid and gid or the currect user (normally root). Many programs (such as passwd) expect to interact with a user through the current TTY, and thus input cannot simply be piped to them after being run with the open function. However, you can use the file handle that this function returns to read and write to such programs as though you were supplying user input and viewing output usually send to the user.

pty_process_exec actually returns both a file handle and the PID of the started process, so that you can kill it if necessary. The standard wait_for function can be used to interact with the command, as this example shows : &foreign_require("proc", "proc-lib.pl"); ($fh, $pid) = &proc::pty_process_exec("passwd $username"); while(1) { $rv = &wait_for($fh, "password:"); if ($rv == 0) { &sysprint($fh, $password); } else { last; } } close($fh);

proc::pty_process_exec_logged(command, [uid, gid]) Just like pty_process_exec, but records the command for later logging when webmin_log is called.

proc::list_processes([pid]) Returns a list of all processes currently running on the system, or just the details of a single process if the pid parameter is supplied. Each element of the returned array is a reference to a hash containing at least the following keys :

proc::find_process(name) Searches for processes whose command or arguments match the given name, and returns an array of details of those that match. Each element is a hash reference with the same members as the list_processes function.

Remote Procedure Calls

Webmin versions 0.82 and above has several common functions for executing code on remote Webmin servers. They are used by some of the standard modules (such as those in the Cluster category) to control multiple servers from a single interface, and may be useful in your own modules as well. These functions, all of which have names starting with remote_, let you call functions, evaluation Perl code, and transfer data to and from other system running Webmin.

Before a 'master' server can make RPC calls to a remote host, it must be registered in the Webmin Servers Index module on the master system. The Link type field must be set to Login via Webmin and a username and password entered. The user specified should be root or admin, as others are not by default allowed to accept RPC calls.

RPC is usually used to call functions in other modules on a remote system, or common functions. This is done with the remote_foreign_call function, but before it can be used remote_foreign_require must be called to load the library for the module that you want to call. This is very similar to calling functions in other local modules with the foreign_ functions, explained in the "Functions in Other Modules" section.

A piece of code that edits a user on a remote system might look like :

$server = "www.example.com";

$user = "joe";

&remote_foreign_require($server, "useradmin", "user-lib.pl");

@users = &remote_foreign_call($server, "useradmin", "list_users");

($joe) = grep { $_->{'user'} eq $user } @users;

if ($joe) {

$joe->{'real'} = "Joe Bloggs";

&remote_foreign_call($server, "useradmin", "modify_user", $joe, $joe);

}

Of course, you need to be familiar with the available functions in other modules, and also to be sure that the module that you want to call is actually installed and of the right version.

All parameters passed to remote functions are converted to a serialized text form for transfer to the remote server, and any return value is also sent back in serialized form. The common functions serialize_variable and unserialize_variable are used, but the process is hidden from both the caller and the remote function - they only see scalars and references in their original format. One thing to look out for is circular references though - trying to send a structure that contains links to itself (such as a doubly-linked list) will fail due to the shortcomings of the serialize_variable function. Also, try to avoid using extremely large parameters, such as strings over 1 MB in size, as serialization may make them massive.

Parameters that are references to hashes, arrays or scalars that would normally be filled in by the function will not be transferred properly. For example, the read_file function normally fills in the hash referenced by its second argument with the contents of a file. This will not work when it is called remotely, as all parameters and anything that they refer to are 'copied' to the other system.

The remote_eval function can be used to execute an arbitrary block of Perl code on a remote system, which allows you to do things that calls to remote functions cannot. It is the only way to call native Perl functions such as unlink, to read and write arbitrary format files, set global variables and properly call functions that set their parameters. Whatever the Perl code evaluates to will be sent back returned by this function. This example shows remote_eval in use :

$data = &remote_eval($server, "useradmin",

"rename('/etc/foo', '/etc/bar');\n".

"local \%data;\n".

"&read_file('/etc/bar', \\%data);\n".

"return \\%data;\n");

&write_file('/etc/foo', $data);

As you can see, proper quoting is necessary when constructing the Perl code string, so that any variable symbols (such as $, % and @) are escape, as is the \ character. The second module parameter to remote_eval can be set to undef, which indicates that the code should be executed in the global Webmin context, rather than in any module's.

The functions remote_read and remote_write can be used to transfer the contents of an entire file between the master and remote systems. They are must faster than reading in the file and encoding it for use in the remote_foreign_call or remote_eval functions, as the file is transferred un-encoded over a TCP connection.

If your module makes RPC calls, you may want the user to select a system to make calls to from a menu. A list of the names of all those available can be obtained from the Webmin Servers Index module with code like this :

&foreign_require("servers", "servers-lib.pl");

@allservers = &servers::list_servers();

@rpcservers = map { $_->{'host'} } grep { $_->{'user'} } @allservers;

In addition, all of the remote_ functions will accept undef for the server parameter. This indicates that the local system should be used, which never needs to be defined in the Webmin Servers Index module. This is how all of the Cluster category modules can include the this server option in their lists of hosts to manage.

Creating Usermin Modules

Usermin has a very similar architecture to Webmin, and so its modules have an almost identical design to Webmin modules. The main difference is that Usermin is designed to be used by any Unix user on a server to perform tasks that they could perform from the command line. Any third-party Usermin modules should be written with this in mind.

By default, module CGI programs are run as root, just like in Webmin. This is necessary because some tasks (like changing passwords) can only be done as root. However, most Usermin modules do not need super-user privileges and so should call the standard switch_to_remote_user function just after calling init_config , in order to lower privileges to those of the logged-in user.

Usermin module can have global configuration variables that are initially set from the config or config-_ostype_ file in the module directory, and are available in %config. However, these variables are never editable by the user - they can only be set in the Usermin Configuration module of Webmin.

Per-user configurable options are supported though, using a different mechanism. When the standard create_user_config_dirs function is called, the global hash %userconfig will be filled with values from the following sources, with later sources overriding earlier ones :

1. The defaultuconfig file in the module directory This should contain the default options for this module for all users, to be used if no other settings are made by the user or system administrator.
2. The file /etc/webmin/_modulename_/defaultuconfig This contains defaults for the module on this system, as set by the system administrator using the second form in the Usermin Module Configuration page feature in the Usermin Configuration Webmin module.
3. The file _~__username_/.usermin/_modulename_/config This contains options chosen by users themselves.

The editors for the system-wide and per-user configuration variables are defined by the uconfig.info file in the module directory. This file has the exact same format as the config.info file used for Webmin and Usermin global configuration, explained elsewhere in this document.

If you create your own Usermin module, it should be packaged in exactly the same way as a Webmin module (as a .tar or .tar.gz file). However, the module.info file must contain the line usermin=1 so that it cannot be installed into Webmin where it would not work properly.

Copyright © by the contributing authors. All material on Doxfer is the property of the contributing authors.
Ideas, requests, problems regarding Doxfer? Send feedback