INSTALL

   1 [mharc Top] mharc Installation
   2
   3 This document describes how to install and configure mharc on your system.
   4 You should read this document in its entirety before performing any of the
   5 installation steps.
   6
   7 NOTE: If performing an upgrade, make sure to read the NEWS document for any
   8       important release notes.
   9
  10 Table of Contents
  11
  12   * Where to Get Help
  13   * Nomenclature
  14   * Audience
  15   * Introduction
  16   * Dependencies
  17       + Platforms
  18       + Software
  19   * Extracting mharc
  20   * Running install.pl
  21   * Configuration Check
  22   * Editing lists.def
  23   * Defining your crontab
  24   * Web Server Configuration
  25   * Maintenance Operations
  26   * Archive Customizations
  27   * Applying Software Updates
  28   * MH/nmh Support
  29   * Managing List Administration Messages
  30       + Procmail Solution
  31       + mharc-based Solution
  32   * Limitations
  33
  34 ---------------------------------------------------------------------------
  35
  36  Where to Get Help
  37
  38 If you have problems installing mharc, a mailing list exists that provides
  39 a forum for help and general discussions. Information about the list, and
  40 how to subscribe, is provided in Contacts.
  41
  42 If you want professional help to setup your list archives, please send a
  43 message to mhonarc@mhonarc.org with your request.
  44
  45 ---------------------------------------------------------------------------
  46
  47  Nomenclature
  48
  49 Shell examples are rendered as follows:
  50
  51 +-------------------------------------------------------------------------+
  52 |prompt> ls -CF                                                           |
  53 |bin/      etc/        images/  log/      msgid.cache       README        |
  54 |cgi-bin/  .htaccess@  info/    Makefile  NEWS              TODO          |
  55 |COPYING   html/       lib/     mbox/     procmailrc.mharc  VERSION       |
  56 +-------------------------------------------------------------------------+
  57
  58 The text prompt> represents your shell prompt. Text you would type into the
  59 shell is rendered like this: text you enter. Any other text is example
  60 output generated by the computer.
  61
  62 ---------------------------------------------------------------------------
  63
  64  Audience
  65
  66 This document's intended audience is people familiar with working in
  67 Unix-type environments. The following prerequisite knowledge is beneficial:
  68
  69   * Know how to start and use a command-line shell.
  70
  71   * Experienced using any one of the myriad of text editors available on
  72     your system. Vi and emacs are the most common, but some of the GUI
  73     environments provide additional text editors.
  74
  75   * Familiar with cron: a daemon to execute scheduled commands. You should
  76     know how to register crontab entries. If you are not familar with cron,
  77     see the crontab(1) and crontab(5) manpages.
  78
  79 ---------------------------------------------------------------------------
  80
  81  Introduction
  82
  83 The typical usage model for mharc is to have a special user account to
  84 perform all mharc-related duties. The account is subscribed to all mailing
  85 lists you want archived. For example, say the account you use is mailarch
  86 and the mail address for the account is mailarch@example.com. For each list
  87 you archive, the address mailarch@example.com must be subscribed to each
  88 list.
  89
  90 This usage model allows mharc to be independent of the mailing list
  91 management software. If changes are made to list management software, mharc
  92 is unaffected. The model also allows a division of labor on who manages the
  93 lists and who manages the archives.
  94
  95 NOTE: Tips on how to handle list administration messages, like subscribe
  96       confirmations, are provided in Managing List Administration Messages
  97
  98 This document assumes this type of usage model. However, due to the
  99 configurable nature of mharc, alternate usage models are possible.
 100
 101 ---------------------------------------------------------------------------
 102
 103  Dependencies
 104
 105 Platforms
 106
 107 mharc runs on any Unix-like operating system. If using a Win32 system, you
 108 will may need to install Cygwin or equivalent software package providing
 109 the Unix tool set.
 110
 111 Software
 112
 113 mharc requires the following software:
 114
 115   * MHonArc
 116     <http://www.mhonarc.org/>:
 117
 118     MHonArc converts messages into HTML and provides the periodic date and
 119     thread indexes. It also allows customization of archive page layout.
 120     v2.5.12, or later, is recommended.
 121
 122   * Procmail
 123     <http://www.procmail.org/>:
 124
 125     Procmail pre-filters incoming mail into the raw mail archives. Note:
 126     The programs formail and lockfile are needed and they are part of the
 127     standard Procmail distribution, but some Unix distributions may include
 128     these programs in separate packages.
 129
 130   * Namazu
 131     <http://www.namazu.org/>:
 132
 133     Namazu provides searching. mharc takes advantage of Namazu's awareness
 134     of MHonArc message pages to provide useful archive navigational aids.
 135     Note: The program namazu.cgi is needed and is part of the standard
 136     Namazu distribution, but some Unix distribtions may include it in a
 137     separate package.
 138
 139   * Perl
 140     <http://www.perl.com/>:
 141
 142     mharc scripts are written in Perl.
 143
 144   * make:
 145
 146     The make program is not strictly required, but the master Makefile
 147     provides a convenient interface to invoking mharc scripts. GNU make is
 148     recommended (and is sometimes installed as gmake), but other variations
 149     should work. make is generally provided on all Unix-like distributions.
 150
 151 NOTE: The install.pl installation script checks for programs required by
 152       mharc. If it cannot locate a program, it generates a warning message.
 153
 154 If you cannot locate any of the above programs or are not sure what is
 155 installed on your system, contact your system administrator (and while your
 156 at it, you may want to ask your sys admin to install mharc for you :-).
 157
 158 ---------------------------------------------------------------------------
 159
 160  Extracting mharc
 161
 162 NOTE: You should be logged into the archive account when installing mharc.
 163
 164 Extract the tar bundle into any temporary location to start the
 165 installation process. The following command extracts the tar.gz
 166 distribution bundle:
 167
 168 +-------------------------------------------------------------------------+
 169 |prompt> gzip -dc mharc-X.X.X | tar xvf -                                 |
 170 |mharc-X.X.X/                                                             |
 171 |mharc-X.X.X/README                                                       |
 172 |mharc-X.X.X/INSTALL                                                      |
 173 |mharc-X.X.X/NEWS                                                         |
 174 |mharc-X.X.X/COPYING                                                      |
 175 |mharc-X.X.X/TODO                                                         |
 176 |mharc-X.X.X/install.pl                                                   |
 177 |mharc-X.X.X/doc/                                                         |
 178 |...                                                                      |
 179 |...[file list snipped for brevity]...                                    |
 180 |...                                                                      |
 181 +-------------------------------------------------------------------------+
 182
 183 where X.X.X represents the version number of mharc you are extracting.
 184
 185 CAUTION: Do not not use mharc under a priviledged user account, like root,
 186          since it may open up security vulnerabilities.
 187
 188 ---------------------------------------------------------------------------
 189
 190  Running install.pl
 191
 192 After you extract the tar bundle, execute the following commands to run the
 193 mharc installation script:
 194
 195 +-------------------------------------------------------------------------+
 196 |prompt> cd mharc-X.X.X                                                   |
 197 |prompt> perl install.pl                                                  |
 198 +-------------------------------------------------------------------------+
 199
 200 The install.pl program will perform some system checks and prompt you for
 201 an installation location. If performing a new install, install.pl prompts
 202 you for some initial configuration information. The following is an example
 203 session of a new install:
 204
 205 +-------------------------------------------------------------------------+
 206 |Looking for 'make' program... '/usr/bin/gmake'                           |
 207 |Looking for 'tar' program... '/bin/tar'                                  |
 208 |Looking for 'cp' program... '/bin/cp'                                    |
 209 |Looking for 'mkdir' program... '/bin/mkdir'                              |
 210 |Looking for 'pwd' program... '/bin/pwd'                                  |
 211 |Looking for 'mhonarc' program... '/usr/local/bin/mhonarc'                |
 212 |Looking for 'mknmz' program... '/usr/local/bin/mknmz'                    |
 213 |Looking for 'namazu.cgi'... '/usr/local/libexec/namazu.cgi'              |
 214 |Looking for 'procmail' program... '/usr/bin/procmail'                    |
 215 |Looking for 'formail' program... '/usr/bin/formail'                      |
 216 |Looking for 'lockfile' program... '/usr/bin/lockfile'                    |
 217 |                                                                         |
 218 |Pathname to install mharc: /home/mailarch/archives                       |
 219 |"/home/mailarch/archives" does not exist, create? ['y']                  |
 220 |Copying files into "/home/mailarch/archives"...                          |
 221 |Copying files into "/home/mailarch/archives/html"...                     |
 222 |Copying files into "/home/mailarch/archives/cgi-bin"...                  |
 223 |Determine MHonArc library path from '/usr/local/bin/mhonarc'... \        |
 224 |        '/usr/local/lib/perl5/site_perl/5.6.1'                           |
 225 |                                                                         |
 226 |Root URL for archives                                                    |
 227 |(You can set this later in lib/config.sh): /archives                     |
 228 |Would you like to edit "lib/config.sh" with "vi"? ['y']                  |
 229 |... [edit session not shown] ...                                         |
 230 |                                                                         |
 231 |Would you like to edit "lib/lists.def" with "vi"? ['y']                  |
 232 |... [edit session not shown] ...                                         |
 233 |                                                                         |
 234 |You are using MHonArc v2.6.7                                             |
 235 |Applying configuration (this may take awhile)...                         |
 236 | ----------------------------------------------------------------------- |
 237 || Please read the mharc installation document to finish the installation |
 238 || process.  A copy is located at:                                        |
 239 || /home/mailarch/archives/doc/install.html                               |
 240 | ----------------------------------------------------------------------- |
 241 +-------------------------------------------------------------------------+
 242
 243 When prompted to edit a file, the editor used is taken from the EDITOR
 244 environment variable. If the environment variable is not set, vi is used.
 245 You are not required to edit files like lib/config.sh and lib/lists.def
 246 during install.pl execution. These files can be edited later as described
 247 in Checking Configuration and Editing lists.def.
 248
 249 NOTE: The installation script provides diagnostics warning you of any
 250       conditions that may prevent mharc from operating properly. Each
 251       warning message provides you with information on what you can do to
 252       fix the problem.
 253
 254 NOTE: The MHonArc version shown may be different than what is shown above.
 255       The version shown should reflect the version of MHonArc found on your
 256       system.
 257
 258 TIP: If you get stuck in vi, you can type the following key strokes to
 259      abandon any changes and exit: <Esc>:q!<Enter>, where <Esc> represents
 260      the Esc key and <Enter> represents the Enter key. Type the following
 261      if you want to save any changes before exiting: <Esc>:wq<Enter>.
 262
 263 ---------------------------------------------------------------------------
 264
 265  Configuration Check
 266
 267 Change your current working directory to the location you installed mharc:
 268
 269 +-------------------------------------------------------------------------+
 270 |prompt> cd /home/mailarch/archives                                       |
 271 +-------------------------------------------------------------------------+
 272
 273 Replace /home/mailarch/archives with the pathname location you specified
 274 when running install.pl.
 275
 276 Examine the lib/config.sh mharc configuration file created by install.pl
 277 and make any edits as needed. Run the following command to have your
 278 changes applied:
 279
 280 +-------------------------------------------------------------------------+
 281 |prompt> make configure                                                   |
 282 |./bin/apply-config -verbose                                              |
 283 |Processing "lib/common.mrc.in"                                           |
 284 |Processing "lib/mrc/_nothread.mrc.in"                                    |
 285 |Processing "lib/mrc/_logo.mrc.in"                                        |
 286 |...                                                                      |
 287 |... [files processed snipped for brevity] ...                            |
 288 |...                                                                      |
 289 |./bin/mhonarc-check                                                      |
 290 |You are using MHonArc v2.6.7                                             |
 291 |=============================================================            |
 292 |* Make sure to rerun 'make configure' when you change                    |
 293 |* lib/config.sh or change a .in template file.                           |
 294 |=============================================================            |
 295 +-------------------------------------------------------------------------+
 296
 297 NOTE: If you edited lib/config.sh while running install.pl, then any
 298       changes you made then would have already been applied.
 299
 300 In general, when you make changes to lib/config.sh, make sure to run make
 301 configure.
 302
 303 NOTE: Make sure to review all variable settings in lib/config.sh. Proper
 304       values are critical for the archiving system to work properly.
 305
 306 ---------------------------------------------------------------------------
 307
 308  Editing lists.def
 309
 310 Once you have finished the main configuration step, you need to define the
 311 lists you want archived. To do this, you edit lib/lists.def. The syntax of
 312 the file is documented in the mk-procmailrc manpage. After editing, run the
 313 following command:
 314
 315 +-------------------------------------------------------------------------+
 316 |prompt> make                                                             |
 317 |You are using MHonArc v2.6.7                                             |
 318 |./bin/mk-procmailrc                                                      |
 319 +-------------------------------------------------------------------------+
 320
 321 A filed called procmailrc.mharc will be created that is used during the
 322 processing of incoming mail. Anytime, you make changes to lib/lists.def,
 323 rerun make to regenerate procmailrc.mharc.
 324
 325 ---------------------------------------------------------------------------
 326
 327  Defining your crontab
 328
 329 To get automatic processing of your archives, you must edit the account's
 330 crontab. The file etc/crontab serves as a template of the crontab entries
 331 needed. To register the file as your crontab, do the following:
 332
 333 +-------------------------------------------------------------------------+
 334 |prompt> crontab etc/crontab                                              |
 335 +-------------------------------------------------------------------------+
 336
 337 CAUTION: If the account has existing crontab entries, you should run the
 338          following command instead:
 339
 340          +----------------------------------------------------------------+
 341          |prompt> crontab -e                                              |
 342          +----------------------------------------------------------------+
 343
 344          And copy the entries in etc/crontab into the account's crontab.
 345
 346 To customize the mharc-related crontab entries, you can edit etc/crontab.in
 347 and run
 348
 349 +-------------------------------------------------------------------------+
 350 |prompt> make configure                                                   |
 351 +-------------------------------------------------------------------------+
 352
 353 to create a new etc/crontab file suitable for copying into the account's
 354 crontab.
 355
 356 NOTE: If your system does not support cron, or you are unable to register
 357       the execution of programs at periodic intervals, you can manually
 358       invoke archive processing by using the various makefile targets shown
 359       in Maintenance Operations.
 360
 361 ---------------------------------------------------------------------------
 362
 363  Web Server Configuration
 364
 365 NOTE: This information provided in the section is specific to the Apache
 366       httpd server.
 367
 368 The file etc/apache.conf provides sample configuration directives for the
 369 Apache HTTP server to control access to your archives. If the default
 370 settings are not sufficient for your needs, you can edit etc/apache.conf.in
 371 and run
 372
 373 +-------------------------------------------------------------------------+
 374 |prompt> make configure                                                   |
 375 +-------------------------------------------------------------------------+
 376
 377 to generate a new etc/apache.conf that can be used in your Apache server
 378 configuration.
 379
 380 If you are on a system where you do not have access to Apache's main server
 381 configuration file, a etc/.htaccess can be used to provide local
 382 configuration settings.
 383
 384 To use this file, copy the generated etc/.htaccess file to the root of the
 385 installation when make configure is done, or even better, create a symlink
 386 to it by executing the following command from the installation root:
 387
 388 +-------------------------------------------------------------------------+
 389 |prompt> ln -s ./etc/.htaccess                                            |
 390 +-------------------------------------------------------------------------+
 391
 392 With the symlink, you do not have to re-copy the file each time you make
 393 changes.
 394
 395 Make sure Apache allows the execution of CGI programs in the cgi-bin
 396 directory. The etc/apache.conf template should allows this via a
 397 ScriptAlias directive.
 398
 399 An alternative is to have filenames with the extension ".cgi" handled by
 400 the cgi-script handler as follows:
 401
 402 AddHandler cgi-script .cgi
 403
 404 NOTE: If you are using the .htaccess method to control access to your
 405       files, you may need to create a .htaccess in the cgi-bin directory
 406       with the following settings:
 407
 408       Options +ExecCGI
 409
 410 ---------------------------------------------------------------------------
 411
 412  Maintenance Operations
 413
 414 Manual maintenance can be done via the Makefile provided. If you run the
 415 command,
 416
 417 +-------------------------------------------------------------------------+
 418 |prompt> make help                                                        |
 419 |Targets available:                                                       |
 420 |  (default)     Generate procmailrc.mharc from ./lib/lists.def.          |
 421 |  configure:    Apply ./lib/config.sh settings.                          |
 422 |  disable:      Disable automated processing of new messages.            |
 423 |  editidx:      Edit all mhonarc archive pages.                          |
 424 |  editidxonly:  Edit only mhonarc archive index pages.                   |
 425 |  editrootidx:  Edit only top period index pages.                        |
 426 |  enable:       Enable automated processing of new messages.             |
 427 |  help:         This message.                                            |
 428 |  readmail:     Process mail spool.                                      |
 429 |  rebuild:      Rebuild archives from raw message data.                  |
 430 |  rootidx:      Regenerated top index for archives.                      |
 431 |...                                                                      |
 432 +-------------------------------------------------------------------------+
 433
 434 You will get a summary of available targets. Targets exist to manually
 435 invoke the mail spool processing, to recreate the entire HTML archives, and
 436 other administrative tasks. For example, to invoke the processing of any
 437 incoming mail, do the following:
 438
 439 +-------------------------------------------------------------------------+
 440 |prompt> make readmail                                                    |
 441 |... [output clipped] ...                                                 |
 442 +-------------------------------------------------------------------------+
 443
 444 Some targets will disable auto-message processing. Targets that do this
 445 will output the following message:
 446
 447 +-------------------------------------------------------------------------+
 448 |=============================================================            |
 449 |!!! Auto-archive processing is DISABLED !!!                              |
 450 |=============================================================            |
 451 +-------------------------------------------------------------------------+
 452
 453 You can run:
 454
 455 +-------------------------------------------------------------------------+
 456 |prompt> make enable                                                      |
 457 |=============================================================            |
 458 |!!! Auto-archive processing is ENABLED !!!                               |
 459 |=============================================================            |
 460 +-------------------------------------------------------------------------+
 461
 462 to re-enable auto-message processing.
 463
 464 ---------------------------------------------------------------------------
 465
 466  Archive Customizations
 467
 468 Files that control archive appearance are controled by template files with
 469 the extension ".in". It is recommended to edit the ".in" version of files
 470 and execute the make configure command to apply your changes.
 471
 472 NOTE: You must run make configure to have mharc recognize any changes made
 473       to a template file.
 474
 475 The ".in.dist" files are versions of the templates as defined by the base
 476 distribution. These will be overwritten when updating the software and
 477 mainly serve as a basis for your custom template files. If you ever you
 478 want to revert back to the ".in.dist" version of a file, just remove the
 479 ".in" version and rerun make configure.
 480
 481 The main MHonArc resource file is lib/common.mrc. To make changes, edit lib
 482 /common.mrc.in and run
 483
 484 +-------------------------------------------------------------------------+
 485 |prompt> make configure                                                   |
 486 +-------------------------------------------------------------------------+
 487
 488 to generate a new lib/common.mrc. You can use @@VARIABLE_NAME@@ references
 489 in lib/common.mrc.in to refer to variables defined in lib/config.sh.
 490 However, this is normally not required since the web-archive program will
 491 pre-define various MHonArc resource variables that reflect settings in lib/
 492 config.sh. See the MHonArc documentation for more information on how to
 493 edit MHonArc resource files.
 494
 495 TIP: To make the maintenance of MHonArc resource settings easier,
 496      especially during mharc upgrades, set the MHA_RC variable in lib/
 497      config.sh to something like the following:
 498
 499      # Pathname to main MHonArc resource file.
 500      MHA_RC=$SW_ROOT/lib/default.mrc
 501
 502      Then create the file $SW_ROOT/lib/default.mrc.in (make note that the
 503      file ends with a ".in" extension) with the following contents:
 504
 505      <Include>
 506      @@SW_ROOT@@/lib/common.mrc
 507      </Include>
 508
 509      <!-- ... customized settings here ... -->
 510
 511      And run:
 512
 513      +--------------------------------------------------------------------+
 514      |prompt> make configure                                              |
 515      +--------------------------------------------------------------------+
 516
 517      Anytime you want to make MHonArc resource changes, make sure to edit
 518      $SW_ROOT/lib/default.mrc.in and rerun make configure.
 519
 520      When you upgrade mharc, and mharc contains a new improved lib/
 521      common.mrc.in.dist, and you want the new settings to be applied to
 522      your archives, you can do the following:
 523
 524      +--------------------------------------------------------------------+
 525      |prompt> rm lib/common.mrc.in                                        |
 526      |prompt> make configure                                              |
 527      +--------------------------------------------------------------------+
 528
 529      Any additional and/or override settings you have in $SW_ROOT/lib/
 530      default.mrc.in are left untouched.
 531
 532      The above avoids performing any messing merging of changes in a new
 533      lib/common.mrc.in.dist to your customized version of lib/
 534      common.mrc.in.
 535
 536 ---------------------------------------------------------------------------
 537
 538  Applying Software Updates
 539
 540 The software is structured to avoid screwing up an existing installation.
 541 When running install.pl, just specify the existing location of your mharc
 542 installation. All the ".in.dist" files will get overwritten, but any ".in"
 543 files will be left untouched inorder to preserve any local customizations.
 544
 545 TIP: If you ever you want to use a new, or revert back to, a ".in.dist"
 546      version of a file, just remove the ".in" version and rerun
 547      make configure.
 548
 549 ---------------------------------------------------------------------------
 550
 551  MH/nmh Support
 552
 553 A program called mh-month-pack exists for sites that have an existing MH/
 554 nmh-based mail filtering setup (either done manually or automatically).
 555 This program can be used to import MH/nmh mail into mharc or to serve as a
 556 replacement to the filter-spool step if you want to continue to use MH/nmh
 557 for handling incoming mail.
 558
 559 If you do this, you will have to modify etc/crontab.in to no longer use
 560 read-mail, but to call mh-month-pack (or some custom script that uses
 561 mh-month-pack) followed by a call to web-archive.
 562
 563 TIP: You may just want to create variant version of read-mail that calls
 564      mh-month-pack instead of filter-spool. Make sure to call your version
 565      something different than read-mail because it will get overridden
 566      during mharc upgrades.
 567
 568 ---------------------------------------------------------------------------
 569
 570  Managing List Administration Messages
 571
 572 Most mailing list management software send out administration message to
 573 subscribers. Examples are subscribe confirmations and subscribed reminders.
 574 A risk exists that these messages can show up in public archives, exposing
 575 sensitive information like subscription passwords.
 576
 577 Two possible solution to the problem are provided:
 578
 579 Procmail Solution
 580
 581 If procmail is your local delivery agent, you can pre-filter all incoming
 582 mail before mharc ever sees it. You can create a .procmailrc file in the
 583 archive accounts's home directory and add rules that forward all list admin
 584 messages to a real person. The .procmailrc may look something like this:
 585
 586 :0
 587 * (^From:(.*[^-a-zA-Z0-9_.])?(majordomo@|mailman-owner@|.*-request@|.*-help@))
 588 ! real-person@example.com
 589
 590 This method is better than the mharc-based solution since it eliminates the
 591 need to poll the archive for any messages and is more secure since any list
 592 administration messages are never in a web accessible location.
 593
 594 mharc-based Solution
 595
 596 The mharc-based solution is to create a special archive to capture admin
 597 messages. For example, the following can be added to the very beginning of
 598 lib/lists.def:
 599
 600 Name: .listsadmin
 601 Description: Lists Admin Messages
 602 From-Address: majordomo@
 603 From-Address: mailman-owner@
 604 From-Address: .*-request@
 605 From-Address: .*-help@
 606 Final: 1
 607
 608 This must occur at the beginning of the file since the filtering rules are
 609 processed from top to bottom. Since the Final option is set, if any message
 610 matches, no further processing is performed.
 611
 612 Since .listsadmin starts with a dot, it will be hidden from the
 613 all-archives list. But since it is possible to for someone to access it
 614 directly, it is best to restrict access to it via HTTP server configuration
 615 (remember to restrict both the raw and html archives).
 616
 617 Now, all you have to do is check the .listsadmin occasionally to see if
 618 anything important has been received.
 619
 620 ---------------------------------------------------------------------------
 621
 622  Limitations
 623
 624   * The archive search forms rely on Javascript to pass around the Namazu
 625     index name since the namazu.cgi program currently does not provide a
 626     namazu template variable for the index name. Hopefully, this limitation
 627     of namazu will be removed in the future so the use of Javascript can be
 628     removed.
 629
 630     If Javascript is disabled, or not supported, in a web client, initial
 631     searches from an archive page will work, but trying to do another
 632     search from the results page will always return no hits.
 633
 634 ---------------------------------------------------------------------------
 635
 636 $Date: 2003/08/09 18:06:40 $
 637 mharc
 638 Copyright © 2002, Earl Hood, earl@earlhood.com
 639