[mharc Top] mharc Installation
This document describes how to install and configure mharc on your system.
You should read this document in its entirety before performing any of the
installation steps.
NOTE: If performing an upgrade, make sure to read the NEWS document for any
important release notes.
Table of Contents
* Where to Get Help
* Nomenclature
* Audience
* Introduction
* Dependencies
+ Platforms
+ Software
* Extracting mharc
* Running install.pl
* Configuration Check
* Editing lists.def
* Defining your crontab
* Web Server Configuration
* Maintenance Operations
* Archive Customizations
* Applying Software Updates
* MH/nmh Support
* Managing List Administration Messages
+ Procmail Solution
+ mharc-based Solution
* Limitations
---------------------------------------------------------------------------
Where to Get Help
If you have problems installing mharc, a mailing list exists that provides
a forum for help and general discussions. Information about the list, and
how to subscribe, is provided in Contacts.
If you want professional help to setup your list archives, please send a
message to mhonarc@mhonarc.org with your request.
---------------------------------------------------------------------------
Nomenclature
Shell examples are rendered as follows:
+-------------------------------------------------------------------------+
|prompt> ls -CF |
|bin/ etc/ images/ log/ msgid.cache README |
|cgi-bin/ .htaccess@ info/ Makefile NEWS TODO |
|COPYING html/ lib/ mbox/ procmailrc.mharc VERSION |
+-------------------------------------------------------------------------+
The text prompt> represents your shell prompt. Text you would type into the
shell is rendered like this: text you enter. Any other text is example
output generated by the computer.
---------------------------------------------------------------------------
Audience
This document's intended audience is people familiar with working in
Unix-type environments. The following prerequisite knowledge is beneficial:
* Know how to start and use a command-line shell.
* Experienced using any one of the myriad of text editors available on
your system. Vi and emacs are the most common, but some of the GUI
environments provide additional text editors.
* Familiar with cron: a daemon to execute scheduled commands. You should
know how to register crontab entries. If you are not familar with cron,
see the crontab(1) and crontab(5) manpages.
---------------------------------------------------------------------------
Introduction
The typical usage model for mharc is to have a special user account to
perform all mharc-related duties. The account is subscribed to all mailing
lists you want archived. For example, say the account you use is mailarch
and the mail address for the account is mailarch@example.com. For each list
you archive, the address mailarch@example.com must be subscribed to each
list.
This usage model allows mharc to be independent of the mailing list
management software. If changes are made to list management software, mharc
is unaffected. The model also allows a division of labor on who manages the
lists and who manages the archives.
NOTE: Tips on how to handle list administration messages, like subscribe
confirmations, are provided in Managing List Administration Messages
This document assumes this type of usage model. However, due to the
configurable nature of mharc, alternate usage models are possible.
---------------------------------------------------------------------------
Dependencies
Platforms
mharc runs on any Unix-like operating system. If using a Win32 system, you
will may need to install Cygwin or equivalent software package providing
the Unix tool set.
Software
mharc requires the following software:
* MHonArc
:
MHonArc converts messages into HTML and provides the periodic date and
thread indexes. It also allows customization of archive page layout.
v2.5.12, or later, is recommended.
* Procmail
:
Procmail pre-filters incoming mail into the raw mail archives. Note:
The programs formail and lockfile are needed and they are part of the
standard Procmail distribution, but some Unix distributions may include
these programs in separate packages.
* Namazu
:
Namazu provides searching. mharc takes advantage of Namazu's awareness
of MHonArc message pages to provide useful archive navigational aids.
Note: The program namazu.cgi is needed and is part of the standard
Namazu distribution, but some Unix distribtions may include it in a
separate package.
* Perl
:
mharc scripts are written in Perl.
* make:
The make program is not strictly required, but the master Makefile
provides a convenient interface to invoking mharc scripts. GNU make is
recommended (and is sometimes installed as gmake), but other variations
should work. make is generally provided on all Unix-like distributions.
NOTE: The install.pl installation script checks for programs required by
mharc. If it cannot locate a program, it generates a warning message.
If you cannot locate any of the above programs or are not sure what is
installed on your system, contact your system administrator (and while your
at it, you may want to ask your sys admin to install mharc for you :-).
---------------------------------------------------------------------------
Extracting mharc
NOTE: You should be logged into the archive account when installing mharc.
Extract the tar bundle into any temporary location to start the
installation process. The following command extracts the tar.gz
distribution bundle:
+-------------------------------------------------------------------------+
|prompt> gzip -dc mharc-X.X.X | tar xvf - |
|mharc-X.X.X/ |
|mharc-X.X.X/README |
|mharc-X.X.X/INSTALL |
|mharc-X.X.X/NEWS |
|mharc-X.X.X/COPYING |
|mharc-X.X.X/TODO |
|mharc-X.X.X/install.pl |
|mharc-X.X.X/doc/ |
|... |
|...[file list snipped for brevity]... |
|... |
+-------------------------------------------------------------------------+
where X.X.X represents the version number of mharc you are extracting.
CAUTION: Do not not use mharc under a priviledged user account, like root,
since it may open up security vulnerabilities.
---------------------------------------------------------------------------
Running install.pl
After you extract the tar bundle, execute the following commands to run the
mharc installation script:
+-------------------------------------------------------------------------+
|prompt> cd mharc-X.X.X |
|prompt> perl install.pl |
+-------------------------------------------------------------------------+
The install.pl program will perform some system checks and prompt you for
an installation location. If performing a new install, install.pl prompts
you for some initial configuration information. The following is an example
session of a new install:
+-------------------------------------------------------------------------+
|Looking for 'make' program... '/usr/bin/gmake' |
|Looking for 'tar' program... '/bin/tar' |
|Looking for 'cp' program... '/bin/cp' |
|Looking for 'mkdir' program... '/bin/mkdir' |
|Looking for 'pwd' program... '/bin/pwd' |
|Looking for 'mhonarc' program... '/usr/local/bin/mhonarc' |
|Looking for 'mknmz' program... '/usr/local/bin/mknmz' |
|Looking for 'namazu.cgi'... '/usr/local/libexec/namazu.cgi' |
|Looking for 'procmail' program... '/usr/bin/procmail' |
|Looking for 'formail' program... '/usr/bin/formail' |
|Looking for 'lockfile' program... '/usr/bin/lockfile' |
| |
|Pathname to install mharc: /home/mailarch/archives |
|"/home/mailarch/archives" does not exist, create? ['y'] |
|Copying files into "/home/mailarch/archives"... |
|Copying files into "/home/mailarch/archives/html"... |
|Copying files into "/home/mailarch/archives/cgi-bin"... |
|Determine MHonArc library path from '/usr/local/bin/mhonarc'... \ |
| '/usr/local/lib/perl5/site_perl/5.6.1' |
| |
|Root URL for archives |
|(You can set this later in lib/config.sh): /archives |
|Would you like to edit "lib/config.sh" with "vi"? ['y'] |
|... [edit session not shown] ... |
| |
|Would you like to edit "lib/lists.def" with "vi"? ['y'] |
|... [edit session not shown] ... |
| |
|You are using MHonArc v2.6.7 |
|Applying configuration (this may take awhile)... |
| ----------------------------------------------------------------------- |
|| Please read the mharc installation document to finish the installation |
|| process. A copy is located at: |
|| /home/mailarch/archives/doc/install.html |
| ----------------------------------------------------------------------- |
+-------------------------------------------------------------------------+
When prompted to edit a file, the editor used is taken from the EDITOR
environment variable. If the environment variable is not set, vi is used.
You are not required to edit files like lib/config.sh and lib/lists.def
during install.pl execution. These files can be edited later as described
in Checking Configuration and Editing lists.def.
NOTE: The installation script provides diagnostics warning you of any
conditions that may prevent mharc from operating properly. Each
warning message provides you with information on what you can do to
fix the problem.
NOTE: The MHonArc version shown may be different than what is shown above.
The version shown should reflect the version of MHonArc found on your
system.
TIP: If you get stuck in vi, you can type the following key strokes to
abandon any changes and exit: :q!, where represents
the Esc key and represents the Enter key. Type the following
if you want to save any changes before exiting: :wq.
---------------------------------------------------------------------------
Configuration Check
Change your current working directory to the location you installed mharc:
+-------------------------------------------------------------------------+
|prompt> cd /home/mailarch/archives |
+-------------------------------------------------------------------------+
Replace /home/mailarch/archives with the pathname location you specified
when running install.pl.
Examine the lib/config.sh mharc configuration file created by install.pl
and make any edits as needed. Run the following command to have your
changes applied:
+-------------------------------------------------------------------------+
|prompt> make configure |
|./bin/apply-config -verbose |
|Processing "lib/common.mrc.in" |
|Processing "lib/mrc/_nothread.mrc.in" |
|Processing "lib/mrc/_logo.mrc.in" |
|... |
|... [files processed snipped for brevity] ... |
|... |
|./bin/mhonarc-check |
|You are using MHonArc v2.6.7 |
|============================================================= |
|* Make sure to rerun 'make configure' when you change |
|* lib/config.sh or change a .in template file. |
|============================================================= |
+-------------------------------------------------------------------------+
NOTE: If you edited lib/config.sh while running install.pl, then any
changes you made then would have already been applied.
In general, when you make changes to lib/config.sh, make sure to run make
configure.
NOTE: Make sure to review all variable settings in lib/config.sh. Proper
values are critical for the archiving system to work properly.
---------------------------------------------------------------------------
Editing lists.def
Once you have finished the main configuration step, you need to define the
lists you want archived. To do this, you edit lib/lists.def. The syntax of
the file is documented in the mk-procmailrc manpage. After editing, run the
following command:
+-------------------------------------------------------------------------+
|prompt> make |
|You are using MHonArc v2.6.7 |
|./bin/mk-procmailrc |
+-------------------------------------------------------------------------+
A filed called procmailrc.mharc will be created that is used during the
processing of incoming mail. Anytime, you make changes to lib/lists.def,
rerun make to regenerate procmailrc.mharc.
---------------------------------------------------------------------------
Defining your crontab
To get automatic processing of your archives, you must edit the account's
crontab. The file etc/crontab serves as a template of the crontab entries
needed. To register the file as your crontab, do the following:
+-------------------------------------------------------------------------+
|prompt> crontab etc/crontab |
+-------------------------------------------------------------------------+
CAUTION: If the account has existing crontab entries, you should run the
following command instead:
+----------------------------------------------------------------+
|prompt> crontab -e |
+----------------------------------------------------------------+
And copy the entries in etc/crontab into the account's crontab.
To customize the mharc-related crontab entries, you can edit etc/crontab.in
and run
+-------------------------------------------------------------------------+
|prompt> make configure |
+-------------------------------------------------------------------------+
to create a new etc/crontab file suitable for copying into the account's
crontab.
NOTE: If your system does not support cron, or you are unable to register
the execution of programs at periodic intervals, you can manually
invoke archive processing by using the various makefile targets shown
in Maintenance Operations.
---------------------------------------------------------------------------
Web Server Configuration
NOTE: This information provided in the section is specific to the Apache
httpd server.
The file etc/apache.conf provides sample configuration directives for the
Apache HTTP server to control access to your archives. If the default
settings are not sufficient for your needs, you can edit etc/apache.conf.in
and run
+-------------------------------------------------------------------------+
|prompt> make configure |
+-------------------------------------------------------------------------+
to generate a new etc/apache.conf that can be used in your Apache server
configuration.
If you are on a system where you do not have access to Apache's main server
configuration file, a etc/.htaccess can be used to provide local
configuration settings.
To use this file, copy the generated etc/.htaccess file to the root of the
installation when make configure is done, or even better, create a symlink
to it by executing the following command from the installation root:
+-------------------------------------------------------------------------+
|prompt> ln -s ./etc/.htaccess |
+-------------------------------------------------------------------------+
With the symlink, you do not have to re-copy the file each time you make
changes.
Make sure Apache allows the execution of CGI programs in the cgi-bin
directory. The etc/apache.conf template should allows this via a
ScriptAlias directive.
An alternative is to have filenames with the extension ".cgi" handled by
the cgi-script handler as follows:
AddHandler cgi-script .cgi
NOTE: If you are using the .htaccess method to control access to your
files, you may need to create a .htaccess in the cgi-bin directory
with the following settings:
Options +ExecCGI
---------------------------------------------------------------------------
Maintenance Operations
Manual maintenance can be done via the Makefile provided. If you run the
command,
+-------------------------------------------------------------------------+
|prompt> make help |
|Targets available: |
| (default) Generate procmailrc.mharc from ./lib/lists.def. |
| configure: Apply ./lib/config.sh settings. |
| disable: Disable automated processing of new messages. |
| editidx: Edit all mhonarc archive pages. |
| editidxonly: Edit only mhonarc archive index pages. |
| editrootidx: Edit only top period index pages. |
| enable: Enable automated processing of new messages. |
| help: This message. |
| readmail: Process mail spool. |
| rebuild: Rebuild archives from raw message data. |
| rootidx: Regenerated top index for archives. |
|... |
+-------------------------------------------------------------------------+
You will get a summary of available targets. Targets exist to manually
invoke the mail spool processing, to recreate the entire HTML archives, and
other administrative tasks. For example, to invoke the processing of any
incoming mail, do the following:
+-------------------------------------------------------------------------+
|prompt> make readmail |
|... [output clipped] ... |
+-------------------------------------------------------------------------+
Some targets will disable auto-message processing. Targets that do this
will output the following message:
+-------------------------------------------------------------------------+
|============================================================= |
|!!! Auto-archive processing is DISABLED !!! |
|============================================================= |
+-------------------------------------------------------------------------+
You can run:
+-------------------------------------------------------------------------+
|prompt> make enable |
|============================================================= |
|!!! Auto-archive processing is ENABLED !!! |
|============================================================= |
+-------------------------------------------------------------------------+
to re-enable auto-message processing.
---------------------------------------------------------------------------
Archive Customizations
Files that control archive appearance are controled by template files with
the extension ".in". It is recommended to edit the ".in" version of files
and execute the make configure command to apply your changes.
NOTE: You must run make configure to have mharc recognize any changes made
to a template file.
The ".in.dist" files are versions of the templates as defined by the base
distribution. These will be overwritten when updating the software and
mainly serve as a basis for your custom template files. If you ever you
want to revert back to the ".in.dist" version of a file, just remove the
".in" version and rerun make configure.
The main MHonArc resource file is lib/common.mrc. To make changes, edit lib
/common.mrc.in and run
+-------------------------------------------------------------------------+
|prompt> make configure |
+-------------------------------------------------------------------------+
to generate a new lib/common.mrc. You can use @@VARIABLE_NAME@@ references
in lib/common.mrc.in to refer to variables defined in lib/config.sh.
However, this is normally not required since the web-archive program will
pre-define various MHonArc resource variables that reflect settings in lib/
config.sh. See the MHonArc documentation for more information on how to
edit MHonArc resource files.
TIP: To make the maintenance of MHonArc resource settings easier,
especially during mharc upgrades, set the MHA_RC variable in lib/
config.sh to something like the following:
# Pathname to main MHonArc resource file.
MHA_RC=$SW_ROOT/lib/default.mrc
Then create the file $SW_ROOT/lib/default.mrc.in (make note that the
file ends with a ".in" extension) with the following contents:
@@SW_ROOT@@/lib/common.mrc
And run:
+--------------------------------------------------------------------+
|prompt> make configure |
+--------------------------------------------------------------------+
Anytime you want to make MHonArc resource changes, make sure to edit
$SW_ROOT/lib/default.mrc.in and rerun make configure.
When you upgrade mharc, and mharc contains a new improved lib/
common.mrc.in.dist, and you want the new settings to be applied to
your archives, you can do the following:
+--------------------------------------------------------------------+
|prompt> rm lib/common.mrc.in |
|prompt> make configure |
+--------------------------------------------------------------------+
Any additional and/or override settings you have in $SW_ROOT/lib/
default.mrc.in are left untouched.
The above avoids performing any messing merging of changes in a new
lib/common.mrc.in.dist to your customized version of lib/
common.mrc.in.
---------------------------------------------------------------------------
Applying Software Updates
The software is structured to avoid screwing up an existing installation.
When running install.pl, just specify the existing location of your mharc
installation. All the ".in.dist" files will get overwritten, but any ".in"
files will be left untouched inorder to preserve any local customizations.
TIP: If you ever you want to use a new, or revert back to, a ".in.dist"
version of a file, just remove the ".in" version and rerun
make configure.
---------------------------------------------------------------------------
MH/nmh Support
A program called mh-month-pack exists for sites that have an existing MH/
nmh-based mail filtering setup (either done manually or automatically).
This program can be used to import MH/nmh mail into mharc or to serve as a
replacement to the filter-spool step if you want to continue to use MH/nmh
for handling incoming mail.
If you do this, you will have to modify etc/crontab.in to no longer use
read-mail, but to call mh-month-pack (or some custom script that uses
mh-month-pack) followed by a call to web-archive.
TIP: You may just want to create variant version of read-mail that calls
mh-month-pack instead of filter-spool. Make sure to call your version
something different than read-mail because it will get overridden
during mharc upgrades.
---------------------------------------------------------------------------
Managing List Administration Messages
Most mailing list management software send out administration message to
subscribers. Examples are subscribe confirmations and subscribed reminders.
A risk exists that these messages can show up in public archives, exposing
sensitive information like subscription passwords.
Two possible solution to the problem are provided:
Procmail Solution
If procmail is your local delivery agent, you can pre-filter all incoming
mail before mharc ever sees it. You can create a .procmailrc file in the
archive accounts's home directory and add rules that forward all list admin
messages to a real person. The .procmailrc may look something like this:
:0
* (^From:(.*[^-a-zA-Z0-9_.])?(majordomo@|mailman-owner@|.*-request@|.*-help@))
! real-person@example.com
This method is better than the mharc-based solution since it eliminates the
need to poll the archive for any messages and is more secure since any list
administration messages are never in a web accessible location.
mharc-based Solution
The mharc-based solution is to create a special archive to capture admin
messages. For example, the following can be added to the very beginning of
lib/lists.def:
Name: .listsadmin
Description: Lists Admin Messages
From-Address: majordomo@
From-Address: mailman-owner@
From-Address: .*-request@
From-Address: .*-help@
Final: 1
This must occur at the beginning of the file since the filtering rules are
processed from top to bottom. Since the Final option is set, if any message
matches, no further processing is performed.
Since .listsadmin starts with a dot, it will be hidden from the
all-archives list. But since it is possible to for someone to access it
directly, it is best to restrict access to it via HTTP server configuration
(remember to restrict both the raw and html archives).
Now, all you have to do is check the .listsadmin occasionally to see if
anything important has been received.
---------------------------------------------------------------------------
Limitations
* The archive search forms rely on Javascript to pass around the Namazu
index name since the namazu.cgi program currently does not provide a
namazu template variable for the index name. Hopefully, this limitation
of namazu will be removed in the future so the use of Javascript can be
removed.
If Javascript is disabled, or not supported, in a web client, initial
searches from an archive page will work, but trying to do another
search from the results page will always return no hits.
---------------------------------------------------------------------------
$Date: 2003/08/09 18:06:40 $
mharc
Copyright © 2002, Earl Hood, earl@earlhood.com