utf 8 on index pages, rt 1443213
[mharc.git] / INSTALL
CommitLineData
2ea8f66b
IK
1[mharc Top] mharc Installation
2
3This document describes how to install and configure mharc on your system.
4You should read this document in its entirety before performing any of the
5installation steps.
6
7NOTE: If performing an upgrade, make sure to read the NEWS document for any
8 important release notes.
9
10Table of Contents
11
12 * Where to Get Help
13 * Nomenclature
14 * Audience
15 * Introduction
16 * Dependencies
17 + Platforms
18 + Software
19 * Extracting mharc
20 * Running install.pl
21 * Configuration Check
22 * Editing lists.def
23 * Defining your crontab
24 * Web Server Configuration
25 * Maintenance Operations
26 * Archive Customizations
27 * Applying Software Updates
28 * MH/nmh Support
29 * Managing List Administration Messages
30 + Procmail Solution
31 + mharc-based Solution
32 * Limitations
33
34---------------------------------------------------------------------------
35
36 Where to Get Help
37
38If you have problems installing mharc, a mailing list exists that provides
39a forum for help and general discussions. Information about the list, and
40how to subscribe, is provided in Contacts.
41
42If you want professional help to setup your list archives, please send a
43message to mhonarc@mhonarc.org with your request.
44
45---------------------------------------------------------------------------
46
47 Nomenclature
48
49Shell examples are rendered as follows:
50
51+-------------------------------------------------------------------------+
52|prompt> ls -CF |
53|bin/ etc/ images/ log/ msgid.cache README |
54|cgi-bin/ .htaccess@ info/ Makefile NEWS TODO |
55|COPYING html/ lib/ mbox/ procmailrc.mharc VERSION |
56+-------------------------------------------------------------------------+
57
58The text prompt> represents your shell prompt. Text you would type into the
59shell is rendered like this: text you enter. Any other text is example
60output generated by the computer.
61
62---------------------------------------------------------------------------
63
64 Audience
65
66This document's intended audience is people familiar with working in
67Unix-type environments. The following prerequisite knowledge is beneficial:
68
69 * Know how to start and use a command-line shell.
70
71 * Experienced using any one of the myriad of text editors available on
72 your system. Vi and emacs are the most common, but some of the GUI
73 environments provide additional text editors.
74
75 * Familiar with cron: a daemon to execute scheduled commands. You should
76 know how to register crontab entries. If you are not familar with cron,
77 see the crontab(1) and crontab(5) manpages.
78
79---------------------------------------------------------------------------
80
81 Introduction
82
83The typical usage model for mharc is to have a special user account to
84perform all mharc-related duties. The account is subscribed to all mailing
85lists you want archived. For example, say the account you use is mailarch
86and the mail address for the account is mailarch@example.com. For each list
87you archive, the address mailarch@example.com must be subscribed to each
88list.
89
90This usage model allows mharc to be independent of the mailing list
91management software. If changes are made to list management software, mharc
92is unaffected. The model also allows a division of labor on who manages the
93lists and who manages the archives.
94
95NOTE: Tips on how to handle list administration messages, like subscribe
96 confirmations, are provided in Managing List Administration Messages
97
98This document assumes this type of usage model. However, due to the
99configurable nature of mharc, alternate usage models are possible.
100
101---------------------------------------------------------------------------
102
103 Dependencies
104
105Platforms
106
107mharc runs on any Unix-like operating system. If using a Win32 system, you
108will may need to install Cygwin or equivalent software package providing
109the Unix tool set.
110
111Software
112
113mharc requires the following software:
114
115 * MHonArc
116 <http://www.mhonarc.org/>:
117
118 MHonArc converts messages into HTML and provides the periodic date and
119 thread indexes. It also allows customization of archive page layout.
120 v2.5.12, or later, is recommended.
121
122 * Procmail
123 <http://www.procmail.org/>:
124
125 Procmail pre-filters incoming mail into the raw mail archives. Note:
126 The programs formail and lockfile are needed and they are part of the
127 standard Procmail distribution, but some Unix distributions may include
128 these programs in separate packages.
129
130 * Namazu
131 <http://www.namazu.org/>:
132
133 Namazu provides searching. mharc takes advantage of Namazu's awareness
134 of MHonArc message pages to provide useful archive navigational aids.
135 Note: The program namazu.cgi is needed and is part of the standard
136 Namazu distribution, but some Unix distribtions may include it in a
137 separate package.
138
139 * Perl
140 <http://www.perl.com/>:
141
142 mharc scripts are written in Perl.
143
144 * make:
145
146 The make program is not strictly required, but the master Makefile
147 provides a convenient interface to invoking mharc scripts. GNU make is
148 recommended (and is sometimes installed as gmake), but other variations
149 should work. make is generally provided on all Unix-like distributions.
150
151NOTE: The install.pl installation script checks for programs required by
152 mharc. If it cannot locate a program, it generates a warning message.
153
154If you cannot locate any of the above programs or are not sure what is
155installed on your system, contact your system administrator (and while your
156at it, you may want to ask your sys admin to install mharc for you :-).
157
158---------------------------------------------------------------------------
159
160 Extracting mharc
161
162NOTE: You should be logged into the archive account when installing mharc.
163
164Extract the tar bundle into any temporary location to start the
165installation process. The following command extracts the tar.gz
166distribution bundle:
167
168+-------------------------------------------------------------------------+
169|prompt> gzip -dc mharc-X.X.X | tar xvf - |
170|mharc-X.X.X/ |
171|mharc-X.X.X/README |
172|mharc-X.X.X/INSTALL |
173|mharc-X.X.X/NEWS |
174|mharc-X.X.X/COPYING |
175|mharc-X.X.X/TODO |
176|mharc-X.X.X/install.pl |
177|mharc-X.X.X/doc/ |
178|... |
179|...[file list snipped for brevity]... |
180|... |
181+-------------------------------------------------------------------------+
182
183where X.X.X represents the version number of mharc you are extracting.
184
185CAUTION: Do not not use mharc under a priviledged user account, like root,
186 since it may open up security vulnerabilities.
187
188---------------------------------------------------------------------------
189
190 Running install.pl
191
192After you extract the tar bundle, execute the following commands to run the
193mharc installation script:
194
195+-------------------------------------------------------------------------+
196|prompt> cd mharc-X.X.X |
197|prompt> perl install.pl |
198+-------------------------------------------------------------------------+
199
200The install.pl program will perform some system checks and prompt you for
201an installation location. If performing a new install, install.pl prompts
202you for some initial configuration information. The following is an example
203session of a new install:
204
205+-------------------------------------------------------------------------+
206|Looking for 'make' program... '/usr/bin/gmake' |
207|Looking for 'tar' program... '/bin/tar' |
208|Looking for 'cp' program... '/bin/cp' |
209|Looking for 'mkdir' program... '/bin/mkdir' |
210|Looking for 'pwd' program... '/bin/pwd' |
211|Looking for 'mhonarc' program... '/usr/local/bin/mhonarc' |
212|Looking for 'mknmz' program... '/usr/local/bin/mknmz' |
213|Looking for 'namazu.cgi'... '/usr/local/libexec/namazu.cgi' |
214|Looking for 'procmail' program... '/usr/bin/procmail' |
215|Looking for 'formail' program... '/usr/bin/formail' |
216|Looking for 'lockfile' program... '/usr/bin/lockfile' |
217| |
218|Pathname to install mharc: /home/mailarch/archives |
219|"/home/mailarch/archives" does not exist, create? ['y'] |
220|Copying files into "/home/mailarch/archives"... |
221|Copying files into "/home/mailarch/archives/html"... |
222|Copying files into "/home/mailarch/archives/cgi-bin"... |
223|Determine MHonArc library path from '/usr/local/bin/mhonarc'... \ |
224| '/usr/local/lib/perl5/site_perl/5.6.1' |
225| |
226|Root URL for archives |
227|(You can set this later in lib/config.sh): /archives |
228|Would you like to edit "lib/config.sh" with "vi"? ['y'] |
229|... [edit session not shown] ... |
230| |
231|Would you like to edit "lib/lists.def" with "vi"? ['y'] |
232|... [edit session not shown] ... |
233| |
234|You are using MHonArc v2.6.7 |
235|Applying configuration (this may take awhile)... |
236| ----------------------------------------------------------------------- |
237|| Please read the mharc installation document to finish the installation |
238|| process. A copy is located at: |
239|| /home/mailarch/archives/doc/install.html |
240| ----------------------------------------------------------------------- |
241+-------------------------------------------------------------------------+
242
243When prompted to edit a file, the editor used is taken from the EDITOR
244environment variable. If the environment variable is not set, vi is used.
245You are not required to edit files like lib/config.sh and lib/lists.def
246during install.pl execution. These files can be edited later as described
247in Checking Configuration and Editing lists.def.
248
249NOTE: The installation script provides diagnostics warning you of any
250 conditions that may prevent mharc from operating properly. Each
251 warning message provides you with information on what you can do to
252 fix the problem.
253
254NOTE: The MHonArc version shown may be different than what is shown above.
255 The version shown should reflect the version of MHonArc found on your
256 system.
257
258TIP: If you get stuck in vi, you can type the following key strokes to
259 abandon any changes and exit: <Esc>:q!<Enter>, where <Esc> represents
260 the Esc key and <Enter> represents the Enter key. Type the following
261 if you want to save any changes before exiting: <Esc>:wq<Enter>.
262
263---------------------------------------------------------------------------
264
265 Configuration Check
266
267Change your current working directory to the location you installed mharc:
268
269+-------------------------------------------------------------------------+
270|prompt> cd /home/mailarch/archives |
271+-------------------------------------------------------------------------+
272
273Replace /home/mailarch/archives with the pathname location you specified
274when running install.pl.
275
276Examine the lib/config.sh mharc configuration file created by install.pl
277and make any edits as needed. Run the following command to have your
278changes applied:
279
280+-------------------------------------------------------------------------+
281|prompt> make configure |
282|./bin/apply-config -verbose |
283|Processing "lib/common.mrc.in" |
284|Processing "lib/mrc/_nothread.mrc.in" |
285|Processing "lib/mrc/_logo.mrc.in" |
286|... |
287|... [files processed snipped for brevity] ... |
288|... |
289|./bin/mhonarc-check |
290|You are using MHonArc v2.6.7 |
291|============================================================= |
292|* Make sure to rerun 'make configure' when you change |
293|* lib/config.sh or change a .in template file. |
294|============================================================= |
295+-------------------------------------------------------------------------+
296
297NOTE: If you edited lib/config.sh while running install.pl, then any
298 changes you made then would have already been applied.
299
300In general, when you make changes to lib/config.sh, make sure to run make
301configure.
302
303NOTE: Make sure to review all variable settings in lib/config.sh. Proper
304 values are critical for the archiving system to work properly.
305
306---------------------------------------------------------------------------
307
308 Editing lists.def
309
310Once you have finished the main configuration step, you need to define the
311lists you want archived. To do this, you edit lib/lists.def. The syntax of
312the file is documented in the mk-procmailrc manpage. After editing, run the
313following command:
314
315+-------------------------------------------------------------------------+
316|prompt> make |
317|You are using MHonArc v2.6.7 |
318|./bin/mk-procmailrc |
319+-------------------------------------------------------------------------+
320
321A filed called procmailrc.mharc will be created that is used during the
322processing of incoming mail. Anytime, you make changes to lib/lists.def,
323rerun make to regenerate procmailrc.mharc.
324
325---------------------------------------------------------------------------
326
327 Defining your crontab
328
329To get automatic processing of your archives, you must edit the account's
330crontab. The file etc/crontab serves as a template of the crontab entries
331needed. To register the file as your crontab, do the following:
332
333+-------------------------------------------------------------------------+
334|prompt> crontab etc/crontab |
335+-------------------------------------------------------------------------+
336
337CAUTION: If the account has existing crontab entries, you should run the
338 following command instead:
339
340 +----------------------------------------------------------------+
341 |prompt> crontab -e |
342 +----------------------------------------------------------------+
343
344 And copy the entries in etc/crontab into the account's crontab.
345
346To customize the mharc-related crontab entries, you can edit etc/crontab.in
347and run
348
349+-------------------------------------------------------------------------+
350|prompt> make configure |
351+-------------------------------------------------------------------------+
352
353to create a new etc/crontab file suitable for copying into the account's
354crontab.
355
356NOTE: If your system does not support cron, or you are unable to register
357 the execution of programs at periodic intervals, you can manually
358 invoke archive processing by using the various makefile targets shown
359 in Maintenance Operations.
360
361---------------------------------------------------------------------------
362
363 Web Server Configuration
364
365NOTE: This information provided in the section is specific to the Apache
366 httpd server.
367
368The file etc/apache.conf provides sample configuration directives for the
369Apache HTTP server to control access to your archives. If the default
370settings are not sufficient for your needs, you can edit etc/apache.conf.in
371and run
372
373+-------------------------------------------------------------------------+
374|prompt> make configure |
375+-------------------------------------------------------------------------+
376
377to generate a new etc/apache.conf that can be used in your Apache server
378configuration.
379
380If you are on a system where you do not have access to Apache's main server
381configuration file, a etc/.htaccess can be used to provide local
382configuration settings.
383
384To use this file, copy the generated etc/.htaccess file to the root of the
385installation when make configure is done, or even better, create a symlink
386to it by executing the following command from the installation root:
387
388+-------------------------------------------------------------------------+
389|prompt> ln -s ./etc/.htaccess |
390+-------------------------------------------------------------------------+
391
392With the symlink, you do not have to re-copy the file each time you make
393changes.
394
395Make sure Apache allows the execution of CGI programs in the cgi-bin
396directory. The etc/apache.conf template should allows this via a
397ScriptAlias directive.
398
399An alternative is to have filenames with the extension ".cgi" handled by
400the cgi-script handler as follows:
401
402AddHandler cgi-script .cgi
403
404NOTE: If you are using the .htaccess method to control access to your
405 files, you may need to create a .htaccess in the cgi-bin directory
406 with the following settings:
407
408 Options +ExecCGI
409
410---------------------------------------------------------------------------
411
412 Maintenance Operations
413
414Manual maintenance can be done via the Makefile provided. If you run the
415command,
416
417+-------------------------------------------------------------------------+
418|prompt> make help |
419|Targets available: |
420| (default) Generate procmailrc.mharc from ./lib/lists.def. |
421| configure: Apply ./lib/config.sh settings. |
422| disable: Disable automated processing of new messages. |
423| editidx: Edit all mhonarc archive pages. |
424| editidxonly: Edit only mhonarc archive index pages. |
425| editrootidx: Edit only top period index pages. |
426| enable: Enable automated processing of new messages. |
427| help: This message. |
428| readmail: Process mail spool. |
429| rebuild: Rebuild archives from raw message data. |
430| rootidx: Regenerated top index for archives. |
431|... |
432+-------------------------------------------------------------------------+
433
434You will get a summary of available targets. Targets exist to manually
435invoke the mail spool processing, to recreate the entire HTML archives, and
436other administrative tasks. For example, to invoke the processing of any
437incoming mail, do the following:
438
439+-------------------------------------------------------------------------+
440|prompt> make readmail |
441|... [output clipped] ... |
442+-------------------------------------------------------------------------+
443
444Some targets will disable auto-message processing. Targets that do this
445will output the following message:
446
447+-------------------------------------------------------------------------+
448|============================================================= |
449|!!! Auto-archive processing is DISABLED !!! |
450|============================================================= |
451+-------------------------------------------------------------------------+
452
453You can run:
454
455+-------------------------------------------------------------------------+
456|prompt> make enable |
457|============================================================= |
458|!!! Auto-archive processing is ENABLED !!! |
459|============================================================= |
460+-------------------------------------------------------------------------+
461
462to re-enable auto-message processing.
463
464---------------------------------------------------------------------------
465
466 Archive Customizations
467
468Files that control archive appearance are controled by template files with
469the extension ".in". It is recommended to edit the ".in" version of files
470and execute the make configure command to apply your changes.
471
472NOTE: You must run make configure to have mharc recognize any changes made
473 to a template file.
474
475The ".in.dist" files are versions of the templates as defined by the base
476distribution. These will be overwritten when updating the software and
477mainly serve as a basis for your custom template files. If you ever you
478want to revert back to the ".in.dist" version of a file, just remove the
479".in" version and rerun make configure.
480
481The main MHonArc resource file is lib/common.mrc. To make changes, edit lib
482/common.mrc.in and run
483
484+-------------------------------------------------------------------------+
485|prompt> make configure |
486+-------------------------------------------------------------------------+
487
488to generate a new lib/common.mrc. You can use @@VARIABLE_NAME@@ references
489in lib/common.mrc.in to refer to variables defined in lib/config.sh.
490However, this is normally not required since the web-archive program will
491pre-define various MHonArc resource variables that reflect settings in lib/
492config.sh. See the MHonArc documentation for more information on how to
493edit MHonArc resource files.
494
495TIP: To make the maintenance of MHonArc resource settings easier,
496 especially during mharc upgrades, set the MHA_RC variable in lib/
497 config.sh to something like the following:
498
499 # Pathname to main MHonArc resource file.
500 MHA_RC=$SW_ROOT/lib/default.mrc
501
502 Then create the file $SW_ROOT/lib/default.mrc.in (make note that the
503 file ends with a ".in" extension) with the following contents:
504
505 <Include>
506 @@SW_ROOT@@/lib/common.mrc
507 </Include>
508
509 <!-- ... customized settings here ... -->
510
511 And run:
512
513 +--------------------------------------------------------------------+
514 |prompt> make configure |
515 +--------------------------------------------------------------------+
516
517 Anytime you want to make MHonArc resource changes, make sure to edit
518 $SW_ROOT/lib/default.mrc.in and rerun make configure.
519
520 When you upgrade mharc, and mharc contains a new improved lib/
521 common.mrc.in.dist, and you want the new settings to be applied to
522 your archives, you can do the following:
523
524 +--------------------------------------------------------------------+
525 |prompt> rm lib/common.mrc.in |
526 |prompt> make configure |
527 +--------------------------------------------------------------------+
528
529 Any additional and/or override settings you have in $SW_ROOT/lib/
530 default.mrc.in are left untouched.
531
532 The above avoids performing any messing merging of changes in a new
533 lib/common.mrc.in.dist to your customized version of lib/
534 common.mrc.in.
535
536---------------------------------------------------------------------------
537
538 Applying Software Updates
539
540The software is structured to avoid screwing up an existing installation.
541When running install.pl, just specify the existing location of your mharc
542installation. All the ".in.dist" files will get overwritten, but any ".in"
543files will be left untouched inorder to preserve any local customizations.
544
545TIP: If you ever you want to use a new, or revert back to, a ".in.dist"
546 version of a file, just remove the ".in" version and rerun
547 make configure.
548
549---------------------------------------------------------------------------
550
551 MH/nmh Support
552
553A program called mh-month-pack exists for sites that have an existing MH/
554nmh-based mail filtering setup (either done manually or automatically).
555This program can be used to import MH/nmh mail into mharc or to serve as a
556replacement to the filter-spool step if you want to continue to use MH/nmh
557for handling incoming mail.
558
559If you do this, you will have to modify etc/crontab.in to no longer use
560read-mail, but to call mh-month-pack (or some custom script that uses
561mh-month-pack) followed by a call to web-archive.
562
563TIP: You may just want to create variant version of read-mail that calls
564 mh-month-pack instead of filter-spool. Make sure to call your version
565 something different than read-mail because it will get overridden
566 during mharc upgrades.
567
568---------------------------------------------------------------------------
569
570 Managing List Administration Messages
571
572Most mailing list management software send out administration message to
573subscribers. Examples are subscribe confirmations and subscribed reminders.
574A risk exists that these messages can show up in public archives, exposing
575sensitive information like subscription passwords.
576
577Two possible solution to the problem are provided:
578
579Procmail Solution
580
581If procmail is your local delivery agent, you can pre-filter all incoming
582mail before mharc ever sees it. You can create a .procmailrc file in the
583archive accounts's home directory and add rules that forward all list admin
584messages to a real person. The .procmailrc may look something like this:
585
586:0
587* (^From:(.*[^-a-zA-Z0-9_.])?(majordomo@|mailman-owner@|.*-request@|.*-help@))
588! real-person@example.com
589
590This method is better than the mharc-based solution since it eliminates the
591need to poll the archive for any messages and is more secure since any list
592administration messages are never in a web accessible location.
593
594mharc-based Solution
595
596The mharc-based solution is to create a special archive to capture admin
597messages. For example, the following can be added to the very beginning of
598lib/lists.def:
599
600Name: .listsadmin
601Description: Lists Admin Messages
602From-Address: majordomo@
603From-Address: mailman-owner@
604From-Address: .*-request@
605From-Address: .*-help@
606Final: 1
607
608This must occur at the beginning of the file since the filtering rules are
609processed from top to bottom. Since the Final option is set, if any message
610matches, no further processing is performed.
611
612Since .listsadmin starts with a dot, it will be hidden from the
613all-archives list. But since it is possible to for someone to access it
614directly, it is best to restrict access to it via HTTP server configuration
615(remember to restrict both the raw and html archives).
616
617Now, all you have to do is check the .listsadmin occasionally to see if
618anything important has been received.
619
620---------------------------------------------------------------------------
621
622 Limitations
623
624 * The archive search forms rely on Javascript to pass around the Namazu
625 index name since the namazu.cgi program currently does not provide a
626 namazu template variable for the index name. Hopefully, this limitation
627 of namazu will be removed in the future so the use of Javascript can be
628 removed.
629
630 If Javascript is disabled, or not supported, in a web client, initial
631 searches from an archive page will work, but trying to do another
632 search from the results page will always return no hits.
633
634---------------------------------------------------------------------------
635
636$Date: 2003/08/09 18:06:40 $
637mharc
638Copyright 2002, Earl Hood, earl@earlhood.com
639