From 2ea8f66bfe7aa5135ae8f38a4278138c1e1dd90a Mon Sep 17 00:00:00 2001 From: Ian Kelling Date: Fri, 17 May 2019 15:45:13 -0400 Subject: [PATCH] add other files worth tracking This is lists.gnu.org without changes as its been for at least many months. Compared to fsf.org, in these files the only change is the bin/ed link and some code in bin/web-archive got removed to show the mbox url next to each month in the archive list. --- .gitignore | 7 +- COPYING | 340 ++++++++++++ INSTALL | 639 +++++++++++++++++++++++ Makefile | 282 ++++++++++ NEWS | 841 ++++++++++++++++++++++++++++++ README | 59 +++ README.FSF | 4 + TODO | 18 + VERSION | 1 + bin/apply-config | 258 +++++++++ bin/compress-files | 180 +++++++ bin/compress-mboxes | 179 +++++++ bin/config-check | 87 ++++ bin/ed | 1 + bin/extract-mesg-date | 188 +++++++ bin/filter-spool | 309 +++++++++++ bin/gc-search-indexes | 192 +++++++ bin/logcmd | 262 ++++++++++ bin/mbox-month-pack | 322 ++++++++++++ bin/mh-month-pack | 285 ++++++++++ bin/mhonarc-check | 95 ++++ bin/mk-procmailrc | 757 +++++++++++++++++++++++++++ bin/mk-search-index | 65 +++ bin/read-mail | 166 ++++++ bin/web-archive | 1153 +++++++++++++++++++++++++++++++++++++++++ html | 1 + mbox | 1 + procmailrc.mailman | 47 ++ 28 files changed, 6738 insertions(+), 1 deletion(-) create mode 100644 COPYING create mode 100644 INSTALL create mode 100644 Makefile create mode 100644 NEWS create mode 100644 README create mode 100644 README.FSF create mode 100644 TODO create mode 100644 VERSION create mode 100755 bin/apply-config create mode 100755 bin/compress-files create mode 100755 bin/compress-mboxes create mode 100755 bin/config-check create mode 120000 bin/ed create mode 100755 bin/extract-mesg-date create mode 100755 bin/filter-spool create mode 100755 bin/gc-search-indexes create mode 100755 bin/logcmd create mode 100755 bin/mbox-month-pack create mode 100755 bin/mh-month-pack create mode 100755 bin/mhonarc-check create mode 100755 bin/mk-procmailrc create mode 100755 bin/mk-search-index create mode 100755 bin/read-mail create mode 100755 bin/web-archive create mode 120000 html create mode 120000 mbox create mode 100644 procmailrc.mailman diff --git a/.gitignore b/.gitignore index 7ab9da8..555baac 100644 --- a/.gitignore +++ b/.gitignore @@ -3,4 +3,9 @@ /.lesshst /.bash* /log -/namazu \ No newline at end of file +/namazu +# debug log file when "The \x22negative number\x22 problem occurred.\n" happens. +# todo: we don't rotate this afaik, we should +/NMZ.bug.info +# files that arent helpful for our purposes to track in git +/doc \ No newline at end of file diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..5b6e7c6 --- /dev/null +++ b/COPYING @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/INSTALL b/INSTALL new file mode 100644 index 0000000..45d4189 --- /dev/null +++ b/INSTALL @@ -0,0 +1,639 @@ +[mharc Top] mharc Installation + +This document describes how to install and configure mharc on your system. +You should read this document in its entirety before performing any of the +installation steps. + +NOTE: If performing an upgrade, make sure to read the NEWS document for any + important release notes. + +Table of Contents + + * Where to Get Help + * Nomenclature + * Audience + * Introduction + * Dependencies + + Platforms + + Software + * Extracting mharc + * Running install.pl + * Configuration Check + * Editing lists.def + * Defining your crontab + * Web Server Configuration + * Maintenance Operations + * Archive Customizations + * Applying Software Updates + * MH/nmh Support + * Managing List Administration Messages + + Procmail Solution + + mharc-based Solution + * Limitations + +--------------------------------------------------------------------------- + + Where to Get Help + +If you have problems installing mharc, a mailing list exists that provides +a forum for help and general discussions. Information about the list, and +how to subscribe, is provided in Contacts. + +If you want professional help to setup your list archives, please send a +message to mhonarc@mhonarc.org with your request. + +--------------------------------------------------------------------------- + + Nomenclature + +Shell examples are rendered as follows: + ++-------------------------------------------------------------------------+ +|prompt> ls -CF | +|bin/ etc/ images/ log/ msgid.cache README | +|cgi-bin/ .htaccess@ info/ Makefile NEWS TODO | +|COPYING html/ lib/ mbox/ procmailrc.mharc VERSION | ++-------------------------------------------------------------------------+ + +The text prompt> represents your shell prompt. Text you would type into the +shell is rendered like this: text you enter. Any other text is example +output generated by the computer. + +--------------------------------------------------------------------------- + + Audience + +This document's intended audience is people familiar with working in +Unix-type environments. The following prerequisite knowledge is beneficial: + + * Know how to start and use a command-line shell. + + * Experienced using any one of the myriad of text editors available on + your system. Vi and emacs are the most common, but some of the GUI + environments provide additional text editors. + + * Familiar with cron: a daemon to execute scheduled commands. You should + know how to register crontab entries. If you are not familar with cron, + see the crontab(1) and crontab(5) manpages. + +--------------------------------------------------------------------------- + + Introduction + +The typical usage model for mharc is to have a special user account to +perform all mharc-related duties. The account is subscribed to all mailing +lists you want archived. For example, say the account you use is mailarch +and the mail address for the account is mailarch@example.com. For each list +you archive, the address mailarch@example.com must be subscribed to each +list. + +This usage model allows mharc to be independent of the mailing list +management software. If changes are made to list management software, mharc +is unaffected. The model also allows a division of labor on who manages the +lists and who manages the archives. + +NOTE: Tips on how to handle list administration messages, like subscribe + confirmations, are provided in Managing List Administration Messages + +This document assumes this type of usage model. However, due to the +configurable nature of mharc, alternate usage models are possible. + +--------------------------------------------------------------------------- + + Dependencies + +Platforms + +mharc runs on any Unix-like operating system. If using a Win32 system, you +will may need to install Cygwin or equivalent software package providing +the Unix tool set. + +Software + +mharc requires the following software: + + * MHonArc + : + + MHonArc converts messages into HTML and provides the periodic date and + thread indexes. It also allows customization of archive page layout. + v2.5.12, or later, is recommended. + + * Procmail + : + + Procmail pre-filters incoming mail into the raw mail archives. Note: + The programs formail and lockfile are needed and they are part of the + standard Procmail distribution, but some Unix distributions may include + these programs in separate packages. + + * Namazu + : + + Namazu provides searching. mharc takes advantage of Namazu's awareness + of MHonArc message pages to provide useful archive navigational aids. + Note: The program namazu.cgi is needed and is part of the standard + Namazu distribution, but some Unix distribtions may include it in a + separate package. + + * Perl + : + + mharc scripts are written in Perl. + + * make: + + The make program is not strictly required, but the master Makefile + provides a convenient interface to invoking mharc scripts. GNU make is + recommended (and is sometimes installed as gmake), but other variations + should work. make is generally provided on all Unix-like distributions. + +NOTE: The install.pl installation script checks for programs required by + mharc. If it cannot locate a program, it generates a warning message. + +If you cannot locate any of the above programs or are not sure what is +installed on your system, contact your system administrator (and while your +at it, you may want to ask your sys admin to install mharc for you :-). + +--------------------------------------------------------------------------- + + Extracting mharc + +NOTE: You should be logged into the archive account when installing mharc. + +Extract the tar bundle into any temporary location to start the +installation process. The following command extracts the tar.gz +distribution bundle: + ++-------------------------------------------------------------------------+ +|prompt> gzip -dc mharc-X.X.X | tar xvf - | +|mharc-X.X.X/ | +|mharc-X.X.X/README | +|mharc-X.X.X/INSTALL | +|mharc-X.X.X/NEWS | +|mharc-X.X.X/COPYING | +|mharc-X.X.X/TODO | +|mharc-X.X.X/install.pl | +|mharc-X.X.X/doc/ | +|... | +|...[file list snipped for brevity]... | +|... | ++-------------------------------------------------------------------------+ + +where X.X.X represents the version number of mharc you are extracting. + +CAUTION: Do not not use mharc under a priviledged user account, like root, + since it may open up security vulnerabilities. + +--------------------------------------------------------------------------- + + Running install.pl + +After you extract the tar bundle, execute the following commands to run the +mharc installation script: + ++-------------------------------------------------------------------------+ +|prompt> cd mharc-X.X.X | +|prompt> perl install.pl | ++-------------------------------------------------------------------------+ + +The install.pl program will perform some system checks and prompt you for +an installation location. If performing a new install, install.pl prompts +you for some initial configuration information. The following is an example +session of a new install: + ++-------------------------------------------------------------------------+ +|Looking for 'make' program... '/usr/bin/gmake' | +|Looking for 'tar' program... '/bin/tar' | +|Looking for 'cp' program... '/bin/cp' | +|Looking for 'mkdir' program... '/bin/mkdir' | +|Looking for 'pwd' program... '/bin/pwd' | +|Looking for 'mhonarc' program... '/usr/local/bin/mhonarc' | +|Looking for 'mknmz' program... '/usr/local/bin/mknmz' | +|Looking for 'namazu.cgi'... '/usr/local/libexec/namazu.cgi' | +|Looking for 'procmail' program... '/usr/bin/procmail' | +|Looking for 'formail' program... '/usr/bin/formail' | +|Looking for 'lockfile' program... '/usr/bin/lockfile' | +| | +|Pathname to install mharc: /home/mailarch/archives | +|"/home/mailarch/archives" does not exist, create? ['y'] | +|Copying files into "/home/mailarch/archives"... | +|Copying files into "/home/mailarch/archives/html"... | +|Copying files into "/home/mailarch/archives/cgi-bin"... | +|Determine MHonArc library path from '/usr/local/bin/mhonarc'... \ | +| '/usr/local/lib/perl5/site_perl/5.6.1' | +| | +|Root URL for archives | +|(You can set this later in lib/config.sh): /archives | +|Would you like to edit "lib/config.sh" with "vi"? ['y'] | +|... [edit session not shown] ... | +| | +|Would you like to edit "lib/lists.def" with "vi"? ['y'] | +|... [edit session not shown] ... | +| | +|You are using MHonArc v2.6.7 | +|Applying configuration (this may take awhile)... | +| ----------------------------------------------------------------------- | +|| Please read the mharc installation document to finish the installation | +|| process. A copy is located at: | +|| /home/mailarch/archives/doc/install.html | +| ----------------------------------------------------------------------- | ++-------------------------------------------------------------------------+ + +When prompted to edit a file, the editor used is taken from the EDITOR +environment variable. If the environment variable is not set, vi is used. +You are not required to edit files like lib/config.sh and lib/lists.def +during install.pl execution. These files can be edited later as described +in Checking Configuration and Editing lists.def. + +NOTE: The installation script provides diagnostics warning you of any + conditions that may prevent mharc from operating properly. Each + warning message provides you with information on what you can do to + fix the problem. + +NOTE: The MHonArc version shown may be different than what is shown above. + The version shown should reflect the version of MHonArc found on your + system. + +TIP: If you get stuck in vi, you can type the following key strokes to + abandon any changes and exit: :q!, where represents + the Esc key and represents the Enter key. Type the following + if you want to save any changes before exiting: :wq. + +--------------------------------------------------------------------------- + + Configuration Check + +Change your current working directory to the location you installed mharc: + ++-------------------------------------------------------------------------+ +|prompt> cd /home/mailarch/archives | ++-------------------------------------------------------------------------+ + +Replace /home/mailarch/archives with the pathname location you specified +when running install.pl. + +Examine the lib/config.sh mharc configuration file created by install.pl +and make any edits as needed. Run the following command to have your +changes applied: + ++-------------------------------------------------------------------------+ +|prompt> make configure | +|./bin/apply-config -verbose | +|Processing "lib/common.mrc.in" | +|Processing "lib/mrc/_nothread.mrc.in" | +|Processing "lib/mrc/_logo.mrc.in" | +|... | +|... [files processed snipped for brevity] ... | +|... | +|./bin/mhonarc-check | +|You are using MHonArc v2.6.7 | +|============================================================= | +|* Make sure to rerun 'make configure' when you change | +|* lib/config.sh or change a .in template file. | +|============================================================= | ++-------------------------------------------------------------------------+ + +NOTE: If you edited lib/config.sh while running install.pl, then any + changes you made then would have already been applied. + +In general, when you make changes to lib/config.sh, make sure to run make +configure. + +NOTE: Make sure to review all variable settings in lib/config.sh. Proper + values are critical for the archiving system to work properly. + +--------------------------------------------------------------------------- + + Editing lists.def + +Once you have finished the main configuration step, you need to define the +lists you want archived. To do this, you edit lib/lists.def. The syntax of +the file is documented in the mk-procmailrc manpage. After editing, run the +following command: + ++-------------------------------------------------------------------------+ +|prompt> make | +|You are using MHonArc v2.6.7 | +|./bin/mk-procmailrc | ++-------------------------------------------------------------------------+ + +A filed called procmailrc.mharc will be created that is used during the +processing of incoming mail. Anytime, you make changes to lib/lists.def, +rerun make to regenerate procmailrc.mharc. + +--------------------------------------------------------------------------- + + Defining your crontab + +To get automatic processing of your archives, you must edit the account's +crontab. The file etc/crontab serves as a template of the crontab entries +needed. To register the file as your crontab, do the following: + ++-------------------------------------------------------------------------+ +|prompt> crontab etc/crontab | ++-------------------------------------------------------------------------+ + +CAUTION: If the account has existing crontab entries, you should run the + following command instead: + + +----------------------------------------------------------------+ + |prompt> crontab -e | + +----------------------------------------------------------------+ + + And copy the entries in etc/crontab into the account's crontab. + +To customize the mharc-related crontab entries, you can edit etc/crontab.in +and run + ++-------------------------------------------------------------------------+ +|prompt> make configure | ++-------------------------------------------------------------------------+ + +to create a new etc/crontab file suitable for copying into the account's +crontab. + +NOTE: If your system does not support cron, or you are unable to register + the execution of programs at periodic intervals, you can manually + invoke archive processing by using the various makefile targets shown + in Maintenance Operations. + +--------------------------------------------------------------------------- + + Web Server Configuration + +NOTE: This information provided in the section is specific to the Apache + httpd server. + +The file etc/apache.conf provides sample configuration directives for the +Apache HTTP server to control access to your archives. If the default +settings are not sufficient for your needs, you can edit etc/apache.conf.in +and run + ++-------------------------------------------------------------------------+ +|prompt> make configure | ++-------------------------------------------------------------------------+ + +to generate a new etc/apache.conf that can be used in your Apache server +configuration. + +If you are on a system where you do not have access to Apache's main server +configuration file, a etc/.htaccess can be used to provide local +configuration settings. + +To use this file, copy the generated etc/.htaccess file to the root of the +installation when make configure is done, or even better, create a symlink +to it by executing the following command from the installation root: + ++-------------------------------------------------------------------------+ +|prompt> ln -s ./etc/.htaccess | ++-------------------------------------------------------------------------+ + +With the symlink, you do not have to re-copy the file each time you make +changes. + +Make sure Apache allows the execution of CGI programs in the cgi-bin +directory. The etc/apache.conf template should allows this via a +ScriptAlias directive. + +An alternative is to have filenames with the extension ".cgi" handled by +the cgi-script handler as follows: + +AddHandler cgi-script .cgi + +NOTE: If you are using the .htaccess method to control access to your + files, you may need to create a .htaccess in the cgi-bin directory + with the following settings: + + Options +ExecCGI + +--------------------------------------------------------------------------- + + Maintenance Operations + +Manual maintenance can be done via the Makefile provided. If you run the +command, + ++-------------------------------------------------------------------------+ +|prompt> make help | +|Targets available: | +| (default) Generate procmailrc.mharc from ./lib/lists.def. | +| configure: Apply ./lib/config.sh settings. | +| disable: Disable automated processing of new messages. | +| editidx: Edit all mhonarc archive pages. | +| editidxonly: Edit only mhonarc archive index pages. | +| editrootidx: Edit only top period index pages. | +| enable: Enable automated processing of new messages. | +| help: This message. | +| readmail: Process mail spool. | +| rebuild: Rebuild archives from raw message data. | +| rootidx: Regenerated top index for archives. | +|... | ++-------------------------------------------------------------------------+ + +You will get a summary of available targets. Targets exist to manually +invoke the mail spool processing, to recreate the entire HTML archives, and +other administrative tasks. For example, to invoke the processing of any +incoming mail, do the following: + ++-------------------------------------------------------------------------+ +|prompt> make readmail | +|... [output clipped] ... | ++-------------------------------------------------------------------------+ + +Some targets will disable auto-message processing. Targets that do this +will output the following message: + ++-------------------------------------------------------------------------+ +|============================================================= | +|!!! Auto-archive processing is DISABLED !!! | +|============================================================= | ++-------------------------------------------------------------------------+ + +You can run: + ++-------------------------------------------------------------------------+ +|prompt> make enable | +|============================================================= | +|!!! Auto-archive processing is ENABLED !!! | +|============================================================= | ++-------------------------------------------------------------------------+ + +to re-enable auto-message processing. + +--------------------------------------------------------------------------- + + Archive Customizations + +Files that control archive appearance are controled by template files with +the extension ".in". It is recommended to edit the ".in" version of files +and execute the make configure command to apply your changes. + +NOTE: You must run make configure to have mharc recognize any changes made + to a template file. + +The ".in.dist" files are versions of the templates as defined by the base +distribution. These will be overwritten when updating the software and +mainly serve as a basis for your custom template files. If you ever you +want to revert back to the ".in.dist" version of a file, just remove the +".in" version and rerun make configure. + +The main MHonArc resource file is lib/common.mrc. To make changes, edit lib +/common.mrc.in and run + ++-------------------------------------------------------------------------+ +|prompt> make configure | ++-------------------------------------------------------------------------+ + +to generate a new lib/common.mrc. You can use @@VARIABLE_NAME@@ references +in lib/common.mrc.in to refer to variables defined in lib/config.sh. +However, this is normally not required since the web-archive program will +pre-define various MHonArc resource variables that reflect settings in lib/ +config.sh. See the MHonArc documentation for more information on how to +edit MHonArc resource files. + +TIP: To make the maintenance of MHonArc resource settings easier, + especially during mharc upgrades, set the MHA_RC variable in lib/ + config.sh to something like the following: + + # Pathname to main MHonArc resource file. + MHA_RC=$SW_ROOT/lib/default.mrc + + Then create the file $SW_ROOT/lib/default.mrc.in (make note that the + file ends with a ".in" extension) with the following contents: + + + @@SW_ROOT@@/lib/common.mrc + + + + + And run: + + +--------------------------------------------------------------------+ + |prompt> make configure | + +--------------------------------------------------------------------+ + + Anytime you want to make MHonArc resource changes, make sure to edit + $SW_ROOT/lib/default.mrc.in and rerun make configure. + + When you upgrade mharc, and mharc contains a new improved lib/ + common.mrc.in.dist, and you want the new settings to be applied to + your archives, you can do the following: + + +--------------------------------------------------------------------+ + |prompt> rm lib/common.mrc.in | + |prompt> make configure | + +--------------------------------------------------------------------+ + + Any additional and/or override settings you have in $SW_ROOT/lib/ + default.mrc.in are left untouched. + + The above avoids performing any messing merging of changes in a new + lib/common.mrc.in.dist to your customized version of lib/ + common.mrc.in. + +--------------------------------------------------------------------------- + + Applying Software Updates + +The software is structured to avoid screwing up an existing installation. +When running install.pl, just specify the existing location of your mharc +installation. All the ".in.dist" files will get overwritten, but any ".in" +files will be left untouched inorder to preserve any local customizations. + +TIP: If you ever you want to use a new, or revert back to, a ".in.dist" + version of a file, just remove the ".in" version and rerun + make configure. + +--------------------------------------------------------------------------- + + MH/nmh Support + +A program called mh-month-pack exists for sites that have an existing MH/ +nmh-based mail filtering setup (either done manually or automatically). +This program can be used to import MH/nmh mail into mharc or to serve as a +replacement to the filter-spool step if you want to continue to use MH/nmh +for handling incoming mail. + +If you do this, you will have to modify etc/crontab.in to no longer use +read-mail, but to call mh-month-pack (or some custom script that uses +mh-month-pack) followed by a call to web-archive. + +TIP: You may just want to create variant version of read-mail that calls + mh-month-pack instead of filter-spool. Make sure to call your version + something different than read-mail because it will get overridden + during mharc upgrades. + +--------------------------------------------------------------------------- + + Managing List Administration Messages + +Most mailing list management software send out administration message to +subscribers. Examples are subscribe confirmations and subscribed reminders. +A risk exists that these messages can show up in public archives, exposing +sensitive information like subscription passwords. + +Two possible solution to the problem are provided: + +Procmail Solution + +If procmail is your local delivery agent, you can pre-filter all incoming +mail before mharc ever sees it. You can create a .procmailrc file in the +archive accounts's home directory and add rules that forward all list admin +messages to a real person. The .procmailrc may look something like this: + +:0 +* (^From:(.*[^-a-zA-Z0-9_.])?(majordomo@|mailman-owner@|.*-request@|.*-help@)) +! real-person@example.com + +This method is better than the mharc-based solution since it eliminates the +need to poll the archive for any messages and is more secure since any list +administration messages are never in a web accessible location. + +mharc-based Solution + +The mharc-based solution is to create a special archive to capture admin +messages. For example, the following can be added to the very beginning of +lib/lists.def: + +Name: .listsadmin +Description: Lists Admin Messages +From-Address: majordomo@ +From-Address: mailman-owner@ +From-Address: .*-request@ +From-Address: .*-help@ +Final: 1 + +This must occur at the beginning of the file since the filtering rules are +processed from top to bottom. Since the Final option is set, if any message +matches, no further processing is performed. + +Since .listsadmin starts with a dot, it will be hidden from the +all-archives list. But since it is possible to for someone to access it +directly, it is best to restrict access to it via HTTP server configuration +(remember to restrict both the raw and html archives). + +Now, all you have to do is check the .listsadmin occasionally to see if +anything important has been received. + +--------------------------------------------------------------------------- + + Limitations + + * The archive search forms rely on Javascript to pass around the Namazu + index name since the namazu.cgi program currently does not provide a + namazu template variable for the index name. Hopefully, this limitation + of namazu will be removed in the future so the use of Javascript can be + removed. + + If Javascript is disabled, or not supported, in a web client, initial + searches from an archive page will work, but trying to do another + search from the results page will always return no hits. + +--------------------------------------------------------------------------- + +$Date: 2003/08/09 18:06:40 $ +mharc +Copyright © 2002, Earl Hood, earl@earlhood.com + diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..d70bc69 --- /dev/null +++ b/Makefile @@ -0,0 +1,282 @@ +##--------------------------------------------------------------------------## +## File: +## $Id: Makefile,v 1.35 2002/10/01 22:49:46 ehood Exp $ +## Description: +## Administrative makefile for mailing list archives. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +.PHONY: default editidx rebuild rootidx dist enable disable readmail \ + help editidxonly release clean distclean configure mhonarc-check \ + doc doc_pod doc_file doc_install + +INSTALL_PL = ./install.pl +APPLY_CONFIG = ./bin/apply-config +COMPRESS_FILES = ./bin/compress-files +COMPRESS_MBOXES = ./bin/compress-mboxes +CONFIG_CHECK = ./bin/config-check +FILTER_SPOOL = ./bin/filter-spool +LOG_CMD = ./bin/logcmd +MBOX_MONTH_PACK = ./bin/mbox-month-pack +MHA_CHECK = ./bin/mhonarc-check +MH_MONTH_PACK = ./bin/mh-month-pack +MK_PROCMAILRC = ./bin/mk-procmailrc +READ_MAIL = ./bin/read-mail +WEB_ARCHIVE = ./bin/web-archive + +PRGS = \ + $(INSTALL_PL) \ + $(LOG_CMD) \ + $(MH_MONTH_PACK) \ + $(MBOX_MONTH_PACK) \ + $(FILTER_SPOOL) \ + $(APPLY_CONFIG) \ + $(COMPRESS_FILES) \ + $(COMPRESS_MBOXES) \ + $(CONFIG_CHECK) \ + $(MHA_CHECK) \ + $(MK_PROCMAILRC) \ + $(READ_MAIL) \ + $(WEB_ARCHIVE) \ + # End PRGS + +CONFIG_SH = ./lib/config.sh +LIST_DEF = ./lib/lists.def + +NO_ARCHIVE = .noarchive +PROCMAIL_RC = procmailrc.mharc + +CHMOD = chmod +CP = cp +FIND = find +GZIP = gzip +MKDIR = mkdir +MV = mv +PERL = perl +RM = rm +TAR = tar +TOUCH = touch + +##--------------------------------------------------------------------------## + +default: procmailrc + +procmailrc: mhonarc-check + $(MK_PROCMAILRC) + +editidx: disable + -$(PERL) $(WEB_ARCHIVE) -verbose -editidx -nosearch $(MLISTS) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is DISABLED !!!" + -@echo "=============================================================" + +editidxonly: disable + -$(PERL) $(WEB_ARCHIVE) -verbose -editidxonly -nosearch $(MLISTS) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is DISABLED !!!" + -@echo "=============================================================" + +editrootidx: disable + -$(PERL) $(WEB_ARCHIVE) -verbose -editrootidx -nosearch $(MLISTS) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is DISABLED !!!" + -@echo "=============================================================" + +rebuild: disable + -$(PERL) $(WEB_ARCHIVE) -verbose -rebuild $(MLISTS) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is DISABLED !!!" + -@echo "=============================================================" + +rootidx: _FORCE + -$(PERL) $(WEB_ARCHIVE) -verbose -editrootidx $(MLISTS) + +enable: _FORCE + @$(RM) -f $(NO_ARCHIVE) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is ENABLED !!!" + -@echo "=============================================================" + +disable: _FORCE + @$(TOUCH) $(NO_ARCHIVE) + -@echo "=============================================================" + -@echo "!!! Auto-archive processing is DISABLED !!!" + -@echo "=============================================================" + +readmail: _FORCE + $(READ_MAIL) -verbose -force + +configure: _FORCE + -@$(MKDIR) -p log mbox html + @if [ ! -f $(CONFIG_SH) ]; then \ + $(CP) $(CONFIG_SH).dist $(CONFIG_SH); \ + $(CHMOD) u+w $(CONFIG_SH); \ + fi + @if [ ! -f $(LIST_DEF) ]; then \ + $(CP) $(LIST_DEF).dist $(LIST_DEF); \ + $(CHMOD) u+w $(LIST_DEF); \ + fi + $(APPLY_CONFIG) -verbose + -$(MHA_CHECK) + -@echo "=============================================================" + -@echo "* Make sure to rerun 'make configure' when you change" + -@echo "* lib/config.sh or change a .in template file." + -@echo "=============================================================" + +mhonarc-check: _FORCE + @$(MHA_CHECK) + +help: _FORCE + -@echo "Targets available:" + -@echo " (default) Generate $(PROCMAIL_RC) from $(LIST_DEF)." + -@echo " configure: Apply $(CONFIG_SH) settings." + -@echo " disable: Disable automated processing of new messages." + -@echo " editidx: Edit all mhonarc archive pages." + -@echo " editidxonly: Edit only mhonarc archive index pages." + -@echo " editrootidx: Edit only top period index pages." + -@echo " enable: Enable automated processing of new messages." + -@echo " help: This message." + -@echo " readmail: Process mail spool." + -@echo " rebuild: Rebuild archives from raw message data." + -@echo " rootidx: Regenerated top index for archives." + -@echo "" + -@echo "NOTE: Targets that modify archives run with debugging enabled." + -@echo "NOTE: 'editidx', 'editidxonly', and 'rebuild' automatically" + -@echo " disable auto-processing. Execute 'enable' target to" + -@echo " reenable." + -@echo "CAUTION: The 'clean' target should not have to be invoked." + -@echo " But if done, it removes all .in generated files. The" + -@echo " 'configure' target can be executed to regenerate" + -@echo " file from templates" + -@echo "WARNING: The 'dist' and 'distclean' targets should only be" + -@echo " invoked IF YOU KNOW WHAT YOU ARE DOING!" + + +# Dummy target to force building of other targets +_FORCE: + +##--------------------------------------------------------------------------## +## The following targets are mainly intended by mharc developers +## and used for creating releases +##--------------------------------------------------------------------------## + +DIST_BUNDLE = mharc-$(_RELEASE_VERSION).tar + +# List of files/dirs to be included in distribtion bundle +# NOTE: The leading '.' is important. +DIST_FILES = \ + ./README \ + ./INSTALL \ + ./NEWS \ + ./COPYING \ + ./TODO \ + ./install.pl \ + ./doc \ + ./bin \ + ./cgi-bin \ + ./etc \ + ./html/.PNM* \ + ./html/stylesheet.css* \ + ./lib \ + ./Makefile \ + # End DIST_FILES + +# List of files/dirs to explicitly exclude from distribtion bundle +# NOTE: The leading '.' is important. +DIST_EXC_FILES = \ + ./etc/release-mharc \ + # End DIST_EXC_FILES + +TAR_EXC_FILE = dist/.dist-exclude + +PERL_PRGS = $(PRGS) + +release: syntax_check dist + +dist: _FORCE + @$(RM) -rf dist + @$(MKDIR) -p dist/mharc + @$(FIND) . -name 'CVS' -print > $(TAR_EXC_FILE) + @$(FIND) . -name '.*.sw*' -print >> $(TAR_EXC_FILE) + @for i in $(DIST_EXC_FILES); do \ + echo $$i >> $(TAR_EXC_FILE); \ + done + @$(TOUCH) INSTALL + @$(TAR) -c \ + -X $(TAR_EXC_FILE) \ + -f - $(DIST_FILES) | (cd dist/mharc; tar xfp -) + @(cd dist/mharc && $(MAKE) distclean) + @(cd dist/mharc && $(MAKE) doc) + @(cd dist/mharc && $(MAKE) doc_pod) + @(cd dist/mharc && echo "$(_RELEASE_VERSION)" > VERSION) + @$(MV) dist/mharc dist/mharc-$(_RELEASE_VERSION) + @(cd dist && $(TAR) cvf $(DIST_BUNDLE) mharc-$(_RELEASE_VERSION)) + @(cd dist && $(GZIP) $(DIST_BUNDLE)) + +syntax_check: _FORCE + @for i in $(PERL_PRGS); do \ + echo "Checking $$i syntax..."; \ + $(PERL) -cw $$i; \ + done + +# doc_pod listed twice to resolve xrefs +doc: doc_pod doc_file doc_install + +doc_pod: _FORCE + -@mkdir -p ./doc/bin ./doc/lib ./doc/file + @for i in $(PRGS); do \ + echo "processing $$i..." ; \ + pod2html --header --infile=$$i \ + --htmlroot=.. \ + --podroot=./doc \ + --podpath='bin:lib' \ + --outfile=./doc/bin/`basename $$i`.html ; \ + done + @for i in `find ./lib/MHArc -name '*.pm' -print`; do \ + echo "processing $$i..." ; \ + pod2html --header \ + --infile=$$i \ + --htmlroot=.. \ + --podroot=./doc \ + --podpath='bin:lib' \ + --outfile=./doc/lib/MHArc::`basename $$i`.html ; \ + done + @$(RM) -f pod2htm*.x~~ + +doc_file: _FORCE + $(CP) -f Makefile ./doc/file/Makefile.txt + $(CP) -f ./lib/common.mrc.in.dist ./doc/file/common.mrc.txt + $(CP) -f ./lib/config.sh.dist ./doc/file/config.sh.txt + $(CP) -f ./lib/lists.def.dist ./doc/file/lists.def.txt + $(CP) -f ./etc/apache.conf.in.dist ./doc/file/apache.conf.txt + $(CP) -f ./etc/crontab.in.dist ./doc/file/crontab.txt + $(CP) -f ./etc/.htaccess.in.dist ./doc/file/htaccess.txt + +doc_install: + w3m -dump -cols 76 ./doc/install.html > INSTALL + +distclean: clean + $(APPLY_CONFIG) -distclean -verbose + $(RM) -f $(CONFIG_SH) $(CONFIG_SH).cache.pl $(LIST_DEF) VERSION INSTALL + $(RM) -rf ./dist ./doc/bin ./doc/lib ./doc/file + +clean: _FORCE + $(RM) -f $(PROCMAIL_RC) + $(APPLY_CONFIG) -clean -verbose + diff --git a/NEWS b/NEWS new file mode 100644 index 0000000..db2d575 --- /dev/null +++ b/NEWS @@ -0,0 +1,841 @@ +=========================================================================== +2003/08/09: v0.7.3 + +Bugs Fixed: + + Bug ID Summary + ------ ------------------------------------------------------------ + 4319 Order of list archive defs in lists.def not reflected in + procmail.mharc. + ------ ------------------------------------------------------------ + + +* lib/common.mrc.in.dist: + . Changed the default value of BOOKMARK-LINK-LABEL custom resource + variable to "Permanent Link". This appears to be more + descriptive and less ambiguous than "Bookmark Link". + The resource variable name will stay the same for compatibility + reasons. + ++ lib/common.mrc.{da,en,pl}.in.dist: + . Localized resource settings for common.mrc. These replace + the mrc/_lang* files. These new files fit with the new Lang + option for lib/lists.def that leverages the new LANG resource + support in MHonArc (v2.6.7 and later) for providing easier + localization management of archives. + + . The Polish localization is new to mharc and contributed by + Bartosz Feski. + +- lib/mrc/_lang_*.mrc.in.dist: + . Removed (see previous item). + +* bin/web-archive: + . Added support for the following lists.def options: + + All-Lists-Name: Archive name to use in all-lists index + Lang: Language of archive + (requires MHonArc v2.6.7 or later). + Hide-From-All-Lists: Do not list archive in all-lists index + + See the mk-procmailrc manpage for more information about these + options. + +* bin/mk-procmailrc: + . Archives can now be defined solely by Procmail-Condition, without + the requirement of defining an Address or From-Address option. + +* etc/crontab.in.dist, + etc/logrotate.conf.in.dist: + . Sample logrotate(8) configuration file and template crontab entry + added to facilitate mharc log file rotation. Logrotate is provided + under most linux distros and is widely used on Unix systems. + +* etc/apache.conf.in.dist: + + Added commented Alias directive for ROOT_URL. + +=========================================================================== +2003/04/07: v0.7.2 + +* bin/apply-config, bin/mh-month-pack: + . Removed 'use Pod::Usage' statement. It is required only when + needed. This should allow script execution on systems that do + not have Pod::Usage installed (but user will not be able to get + help from the command-line). + +* bin/apply-config: + . Minor change which should make apply-config run under older versions + of Perl with older versions of File::Find module. + +* cgi-bin/mnav.cgi.in.dist: + . Https check added so redirect URL sent to client will start with + 'https://' if secure HTTP is being used (thanks to Jonathon Padfield + for catching this). + +* bin/mk-procmailrc: + . Some optimization cleanup on procmailrc generation. + +* bin/web-archive: + . Added -keepsearch option to preserve the search index when + -rebuild is specified. + +* install.pl: + . Added $HOME/libexec, /usr/lib/cgi-bin, and /usr/local/lib/cgi-bin + paths when searching for namazu.cgi. + +=========================================================================== +2002/10/02: v0.7.1 + +* lib/lists.def, bin/mk-procmailrc, bin/web-archive: + . Added support for Check-No-Archive for lists.def. + The author can request no archiving should be done by defining + one of the the following header fields: + + Restrict: no-external-archive + X-no-archive: yes + + If Check-No-Archive is enabled, a message to the list with either + field defined will not be archived. + +* lib/mk-procmailrc: + . Can now specify a mail address to receive all unmatched messages + via the CATCH_ADDRESS config.sh variable. This serves as a + (better?) alternative to the .catch archive. + +* bin/logcmd: + . New script to help provide better logging of crontab commands. + +* etc/crontab.in.dist: + . Entries changed to use new logcmd script for dumping any command + output to @@SW_ROOT@@/log/cron.log. + +* lib/config.sh.in.dist: + . Added CATCH_ADDRESS: Specify mail address to receive all unmatched + messages. If defined, it supercedes $CATCH_ARCHIVE. + +=========================================================================== +2002/09/28: v0.7.0 + +* Extracting the mharc distribution will now generate a directory + called "mharc-X.X.X", where X.X.X is the version of mharc. A new + install script (see next change item) is now provided to install + mharc into the location you desire. + + Existing users should be aware of that you no longer just extract + a distribution in the location of your existing mharc installation + to perform upgrades. You will now extract in a temporary location, + and then use the install script provided (see next change item) + to upgrade an existing installation. + +* install.pl: + . New Perl installation program for mharc. It should make the + installation easier and more automated, and for new installations, + it will preset some config.sh variables. + + NOTE: Not all installation steps can be automated, like crontab + editing, web server configure, etc. See the installation document + for more information on the new installation process. + +* bin/apply-config: + . BUG FIX: Fixed handling of case when no directories are supplied + on the command-line. Cases where HTML_DIR, or other directory + variables, are defined to pathnames not under $SW_ROOT are now + handled. Thanks to Eugene Eric Kim for spotting this problem. + +* bin/filter-spool: + . Updated to use new PROCMAIL, FORMAIL, and LOCKFILE variables + representing the pathnames to the respective programs used by + filter-spool. + +* bin/web-archive: + . BUG FIX: The ALL_LISTS_FILE variable in config.sh was not + supported. web-archive was looking up the wrong variable name. + +* lib/config.sh.dist: + . Added CGI_DIR variable to represent pathname to CGI bin directory + (default value is $SW_ROOT/cgi-bin). Various CGI variables now + reference this variable in the default values. CGI_DIR exists + mainly for use by the new install.pl script. + + . Added PROCMAIL variable to represent pathname to procmail program. + . Added LOCKFILE variable to represent pathname to lockfile program. + . Added FORMAIL variable to represent pathname to formail program. + +=========================================================================== +2002/09/19: v0.6.4 + +* bin/web-archive: + . BUG FIX: Changes to fix backdoor to raw mail messages for + archives with No-Raw-Link enabled. For archives with No-Raw-Link + enabled, a file called ".noraw" will appear for the archives + under the raw mailbox data. This file is used by extract-mesg.cgi + to determine if retrieval of original messages is allowed. + + Also, a .htaccess file will be created to deny access via direct + HTTP access by a client for those that are using the Apache + HTTP server with .htaccess support enabled. If .htaccess is not + supported (or you are using a non-Apache web server), you will need + to modify the server configuration file directly to deny access. + + . Added support for localizing labels in all-lists archive listing + table via new lib/config.sh variables. + +* cgi-bin/extract-mesg.cgi.in.dist: + . BUG FIX: Changes to fix backdoor to raw mail messages for + archives with No-Raw-Link enabled. In order to enable the new + fixed version, you MUST do the following: + + prompt> rm cgi-bin/extract-mesg.cgi.in + prompt> make configure + + . BUG FIX: Fixed regex pattern checking for valid period argument. + Extraction of yearly-based archives should now work properly. + +* lib/config.sh.dist: + . Added the following variables to support localization of all-lists + page (default values shown): + + ALL_LISTS_LABEL_NAME='Name' + ALL_LISTS_LABEL_INDEXES='Current Index' + ALL_LISTS_LABEL_LAST_UPDATED='Last Updated' + ALL_LISTS_LABEL_INFO='[info]' + ALL_LISTS_LABEL_DATE='[Date]' + ALL_LISTS_LABEL_THREADS='[Thread]' + ALL_LISTS_DATE_FORMAT='%Y-%m-%d %H:%M:%S' + +* lib/mrc/_lang_da.mrc.in.dist: + . NEW: Danish localization file. Thanks goto Lars Jorgensen for + the contribution. + +* lib/mrc/_nospam.mrc.in.dist: + . Added definition of FIELDORDER to provide a minimal message header. + This is to minimize the potential of addresses showing up in + unexpected header fields. + +* bin/config-check: + . NEW: New script to dump expanded variable values of lib/config.sh + to standard output. + +* lib/MHArc/Config.pm: + . A Perl cache file is now created to make subsequent loads faster. + Hence, you will see a file called lib/config.sh.cache.pl appear. + This file should not be edited. The module auto-detects when + lib/config.sh is modified. + +* doc/: + . Some documentation fixes and enhancements. + +=========================================================================== +2002/09/15: v0.6.3 + +* Makefile: + . BUG FIX: Fixed bug with 'make configure' where mhonarc-check + should be done after config.sh exists. + +=========================================================================== +2002/09/14: v0.6.2 + +* doc/: + . NEW: HTML documentation now exists, including documentation for + the scripts and modules and an updated, and improved, installation + document. + + Existing mharc users are encouraged to read through the new + documentation, especially the installation document, since some + helpful new information has been added. + +* bin/web-archive: + . Added definition of $NMZ-SUBJECT-QUERY$ resource variable. It + expands to a namazu query string that will find all messages with + the same subject as the current message. + + This variable is referenced in the updated lib/common.mrc.in.dist + resource for use with the new [More on this subject...] link + at the end of the message body. + +* lib/common.mrc.in.dist: + . Added "[More on this subject...]" to message page layout that + will provide a list of archive messages with the same subject as + the current message. This is an alternative navigational aid + to discussion threads since it is not limited to the current + period as thread listings are. + + . Some restructuring to support localization. For an example on how + localization can be done for a language, see + lib/mrc/_lang_en.mrc.in.dist. + +* etc/apache.conf.in.dist, etc/.htaccess.in.dist: + . Added access denial to working/temporary ".newmail" file used by + filter-spool. + . apache.conf directives adjusted to separate cgi-bin into its own + Directory directive to provide better granularity of access. + +* lib/mrc/_lang_en.mrc.in.dist: + . NEW: Includable resource file for English localization + (defaults in common.mrc.in.dist are already in English, so this + file mainly serves as a template for creating resource files + for other languages). + +* lib/mrc/_logo.mrc.in.dist: + . NEW: Includable resource file to allow for the addition of + a logo, or small image, at the top of archive pages. + +* lib/mrc/_nosearch.mrc.in.dist: + . NEW: Includable resource file for archives with searching disabled. + +* lib/mrc/_nospam.mrc.in.dist: + . NEW: Includable resource file for those who want to deter + address harvesters. + +* bin/*: + . Programs in bin directory all perform a check if invoked in a + CGI context. If so, the program will abort execution. + This is done to provide some extra security incase of server + misconfiguration. + +=========================================================================== +2002/09/04: v0.6.1 + +* bin/extract-mesg-date: + . Added check for new MSG_DATE_FIELDS config.sh variable denoting + the message header fields to extract the date from. The + value of MSG_DATE_FIELDS is used if -datefields is not specified. + +* bin/filter-spool: + . If there is no mail in the spool, but .newmail exists, + filter-spool will still process .newmail. .newmail is the + messages copied from the spool when filter-spool is executing. + In the past, even if .newmail exists, no mail in the spool + will cause filter-spool to exit. + +* bin/web-archive: + . Added check for MSG_DATE_FIELDS config.sh variable. If set, + its value will passed to MHonArc via the DATEFIELDS resource. + . Define $MESG-CGI$ resource variable for use within MHonArc + resource files. + +* cgi-bin/extract-mesg.cgi.in.dist: + . Internal changes so its uses new MHArc::CGI utility routines. + . POD added. + +* cgi-bin/mesg.cgi.in.dist: + . NEW: New CGI program to extract a message from an archive based + upon message-id. This allows for persistent URLs to messages + that are immune to archive rebuilds that could change MHonArc + message numbers. + + This CGI is referenced in the updated lib/common.mrc.in.dist + resource file via the $MESG-CGI$ resource variable. It is + used for the new [Bookmark Link] on message pages. + +* cgi-bin/mnav.cgi.in.dist: + . Added some sanity checks to argument input. + . Internal changes so its uses new MHArc::CGI utility routines. + . POD added. + +* lib/common.mrc.in.dist: + . Added "[Bookmark Link]" to message page layout to provide a + persistent link to the message. The link provided makes use + of the new mesg.cgi CGI script. + +* lib/config.sh.dist: + . Added MSG_DATE_FIELDS: List of message header fields to extract + the date for a message. + . Added MESG_CGI: URL to persistent message reference CGI program + (mesg.cgi). + +* lib/MHArc/CGI.pm: + . NEW: Shared CGI utility routines for mharc CGI scripts. + +* lib/MHArc/MailUtil.pm: + . extract_date() routine changed to handle indexed header fields + in date fields argument. + + CAUTION: Only use indexed field specification if you are using + a version of MHonArc *newer than* 2.5.11. MHonArc v2.5.11, + and earlier versions, do not support indexed fields for the + DATEFIELDS resource. + +* lib/MHArc/Namazu.pm: + . NEW: Collection of routines related to Namazu. + +=========================================================================== +2002/08/26: v0.6.0 + +* bin/compress-files, bin/compress-mboxes: + . BUG FIX: Added -mbox-mode to compress-files. This is to fix a bug + with compress-mboxes for archives that have yearly archives. + If a mailbox is in YYYY format, a year is added to -mtime to + determine if the file should be compressed. + +* bin/web-archive: + . NEW: Added auto-generation of all-lists index. The index will + contain the list of archives with links to the latest indexes + and showing the last update time for each archive. + + The all-lists index file can be specified by the -alllistidx + option or ALL_LISTS_INDEX lib/config.sh variable. The default + value is lists.html within the html/ archive directory. + Equivalent options exists to set the URL to the file. + + Run web-archive program with -man option to view the manpage + to get more information about this feature and other related + options. + +* lib/common.mrc.in.dist: + . Several layout changes to provide a nicer appearance to archive + pages. Layout is now controled via stylesheet settings. + Tags have been added with CLASS attributes to allow control + of the visual appearance of pages without having to re-edit + the pages. An initial stylesheet (html/stylesheet.css.in.dist) + has been provided with some default styles (more information + about stylesheet below). + + Attempts have been made to keep pages usable with text-based + browsers. Table-aware browsers like w3m and links, handle + the pages fine. Lynx does not handle the tables well, but + the pages should still be readable and usable with Lynx. + + To see an example of the layout style, you can checkout the + mhonarc.org mailing lists at + . + + If you want to use the new layout, you must delete your local + copy of "lib/common.mrc.in" and run "make configure". + +* html/stylesheet.css.in.dist: + . NEW: This is the main stylesheet controling the appearance + of archive pages generated by MHArc. Stylesheet settings are + based upon the CSS2 specification. The styles are supported by + Netscape 6+, Mozilla, and Mozilla-based browsers. Some styles + appear to not work with IE6, however, the failures of IE6 do not + affect the readability of pages. + +* lib/mrc/_nothread.mrc.in.dist: + . NEW: MHonArc resource file use by archives that have threads + disabled, either by specifying -nothreads in the MHonArc-Options + option in lib/lists.def for a list or by a list-specific resource + file. + + NOTE: There is a feature in MHArc that has not been explicitly + documented that allows you to provide additional + MHonArc resource settings on a per-list archive basis. + In lib/config.sh is a variable called MHA_RC_DIR that + specifies the location of archive-specific resource files. + By default, it is "/lib/mrc". If you add a + file called ".mrc", where "" is the + name of the list as defined in lib/lists.def, MHArc will + pass that file to MHonArc when processing the HTML archive + for the list. + + When creating a list-specific resource file, it is + recommended to create the file + "/lib/mrc/.mrc.in" and then run + "make configure". This allows you to use lib/config.sh + variables (e.g. "@@HTML_URL@@") in your resource file. + + A way to reference _nothread.mrc in a resource file is to have + the following in the ".in" template version of a list resource + file: + + + @@MHA_RC_DIR@@/_nothread.mrc + + + When "make configure" is executed, the "@@MHA_RC_DIR@@" will get + expanded to the pathname location where archive-specific resource + files are kept. + +* lib/config.sh.in.dist: + . Added the following variables: + + # Pathname of archive lists index page. + ALL_LISTS_FILE=$HTML_DIR/lists.html + + # Pathname of header template for archive lists index page. + ALL_LISTS_HEADER=$HTML_DIR/.PNM.all-head + + # Pathname of header template for archive lists index page. + ALL_LISTS_FOOTER=$HTML_DIR/.PNM.all-foot + + # Directory containing informational pages about each list archive. + INFO_DIR=$SW_ROOT/info + + # Base URL containing informational pages about each list archive. + INFO_URL=$ROOT_URL/info + + The INFO_DIR and INFO_URL are used when generating the + all-lists index page. They give the ability for you to create + an informational page for each list archive. Run web-archive + program with -man option to view the manpage add read about the + -infodir and -infourl options for more information. + +* cgi-bin/template/NMZ.head.in.dist, html/.PNM.head.in.dist: + . Minor change to labels for sorting search results by date. + Newer labels should be clearer about the ordering of dates since + previous wording can be ambiguous. + + If you want to use the newer versions, delete + "cgi-bin/template/NMZ.head.in" and "html/.PNM.head.in" and run + "make configure". + +=========================================================================== +2002/07/31: v0.5.1 + +* cgi-bin/extract-mesg.cgi.in.dist: + . Changed returned media-type from message/rfc822 to text/plain. + message/rfc822 is nice since some browsers can render it directly, + but it does open potential XSS HTML email attacks. + + IMPORTANT NOTE: User upgrading are encouraged to delete + "cgi-bin/extract-mesg.cgi.in" and run 'make + configure' after extracting this release. If you + really want the message/rfc822 behavior, you can + edit "cgi-bin/extract-mesg.cgi.in" and redefine + the $message_media_type variable. + +* bin/mk-procmailrc: + . Added -final-dest option: The destination of messages that make + it to the end of the procmailrc. Generally, this option can + be ignored. + + Run mk-procmailrc program with -man option to view the manpage + to get more information about this option. + +* etc/apache.conf.in.dist: + . Security related comments added. Users are encourage to read + if using etc/apache.conf. + +* lib/config.sh.dist: + . Added FINAL_MSG_DESTINATION variable to repesent destination + mailbox for end rule in procmailrc (checked by mk-procmailrc if + -final-dest not specified). + +=========================================================================== +2002/07/24: v0.5.0 + +* bin/mk-procmailrc + . Added -out option that can be used to specify the name of + the procmailrc file to generate. If not specified, the PROCMAILRC + variable in lib/config.sh is used. Otherwise, the default value is + "$SW_ROOT/procmailrc.mharc"; + + IMPORTANT NOTE: This implies that the name of the main procmailrc + used by mharc has changed from + $SW_ROOT/.procmailrc. This is to avoid potential + conflict with environments that use Procmail as a + local delivery agent and MHArc has been extracted + in the home directory of the archiving account. + + If upgrading, all you should have to do is + invoke "make" to create the procmailrc with + the newer filename. Then, you can remove the + old .procmailrc. + +* bin/web-archive: + . BUG FIX: Restored older semantics for $LIST-NAME$: It equals the + list name as represented by the directory name for the list + archives. This change back only has an effect for complimentary + CVS commit archives since the $LIST-NAME$ is used as the search + index name. + + A new resource variable, $LIST-TITLE$, is used to represent the + displayable list name. For CVS commit archives, this variable + will have the "[CVS] " prefix. + + In sum, everywhere the list name is used in links or form input + elements, use $LIST-NAME$. Anywhere else, $LIST-TITLE$ can + be used. + + . Added $EXTRACT-CGI$ resource variable + (see cgi-bin/extract-mesg.cgi.in.dist change entry below for + more information). + +* cgi-bin/extract-mesg.cgi.in.dist: + . NEW: CGI program to extract original raw message. It is intended + to be used in HTML archive message pages to allow the reader to + retrieve the original mail message. + + The file lib/common.mrc.in.dist has been modified to include + an "[Original]" link on messages pages. If you would like this + feature in an existing mharc archive, you will need to edit + your lib/common.mrc.in an add the link. Here is the resource + setting added to lib/common.mrc.in.dist: + + +
+ $BUTTON(PREV)$$BUTTON(NEXT)$$BUTTON(TPREV)$$BUTTON(TNEXT)$[Date Index][Thread Index][Original] + + +* lib/config.sh.dist: + . Added EXTRACT_CGI variable that represents URL to extract-mesg.cgi. + . Added PROCMAILRC variable to define main procmailrc file used + by bin/mk-procmailrc and bin/filter-spool. + . Added ORGMAIL_LOCK_TIMEOUT to set lock timeout used by filter-spool. + . Added LOG_DIR variable to represent location to place log files. + +* lib/common.mrc.in.dist: + . Replaced $LIST-NAME$ with $LIST-TITLE$ in MSGPGBEGIN resource. + +* etc/apache.conf.in.dist, etc/.htaccess.conf.in.dist: + . Added denial of files starting with "procmail". + +* bin/apply-conf: + . Explicitly remove group and other write bit on generated files + that are executable. This helps avoid Apache HTTP server suEXEC + problems with CGI programs. + +* bin/filter-spool: + . Rewritten in Perl. Run program with -man option to view manpage. + +* bin/read-mail: + . Rewritten in Perl. Run program with -man option to view manpage. + +=========================================================================== +2002/07/19: v0.4.0 + +* bin/compress-mboxes: + . FIX: Fixed handling of -n and -debug options. + +* lib/config.sh.dist: + . FIX: Default PROCMAIL_PATH setting uses $SW_ROOT/bin instead of + $HOME/bin. Installations should update local config.sh + to reflect change. + + . Added MHONARC_LIB: Directory pathname to where MHonArc libraries + are installed. If MHonArc libraries are in a location not included + in perl's library search path, this variable should be set to + that location. + + . Added MSGID_CACHE_SIZE: Maximum size, in bytes, of msgid cache + for detecting duplicate messages. + + . Reorganized file to put key variables at top of file and to + provide a comment description before each variable. + +* bin/mbox-month-pack: + . NEW: New utility program for breaking up a mailbox file into + monthly, or yearly, mailbox files. This program can be useful + for importing existing mailbox files into MHArc. Run program with + the -man option to view manpage. + +* bin/mk-procmailrc: + . Added support for "From-Address: " option in lists.def. This + option allows you to denote a list by the "From: " field of + messages. Run mk-procmailrc with -man option for more information. + + . "Cleaned-up" the procmailrc file generated. The "Extraneous + copy flag" warnings should be gone. Also, rules changed to use + new bin/extract-mesg-date to determine which raw mailbox to file + to instead of the current time. This insures that messages go + into the proper period mailbox regardless of when incoming mail + is processed. + +* bin/extract-mesg-date: + . NEW: New utility to print out the date of a message. This + program is mainly used in newer procmailrc format generated + by mk-procmailrc. + +=========================================================================== +2002/07/08: v0.3.0 + +* mk-procmailrc: + . Can now specify name of catch archive and if catch archive should + be disabled via the CATCH_ARCHIVE and DISABLE_CATCH_ARCHIVE config.sh + variables. + + . POD added to script along with -help and -man options to display + usage information. + +* web-archive: + . Added check for and + comment declarations in period index header/footer templates: + .PNM.head.in and .PNM.foot.in. If an archive is designated + to not have searching enabled, any markup between these two + comments will be stripped out when generating the period index. + + If upgrading, and if you are using the default .PNM.*.in files, + you will need to delete them so the newer versions with the above + comments will be applied. If you have customized versions of + .PNM.*.in files, you will need to add the above comments around + the search form markup if you want the form to not appear in the + top period indexes for non-searchable archives. + + . CVS commit archive of a list properly uses list options defined + in lib/lists.def. + + . Short title of CVS commit archives has changed from + ".CVS" to "[CVS] ". + +* bin/compress-files: + . Add POD. Execute './bin/compress-files -man' to view + documentation. + +* bin/compress-mboxes: + . Converted to Perl. POD added along with some command-line options. + Execute './bin/compress-mboxes -man' to view documentation. + +* bin/gc-search-indexes + . Converted to Perl. POD added along with some command-line options. + Execute './bin/gc-search-indexes -man' to view documentation. + +* etc/apache.conf.in.dist: + . Updated to add directives to deny access to various files + that you may not want visible to the public. + +* etc/.htaccess.in.dist: + . NEW: Contains the same directives added to + etc/apache.conf.in.dist. This file is useful if you do not have + write access to Apache's main configuration file. + + The .htaccess file generated after 'make configure' can be copied + into the root install directory, or better, create a symlink + to it from the root by executing the following command in the + root install directory. + + ln -s ./etc/.htaccess + +* Makefile: + . Added editrootidx target: Rebuild top period indexes only. + +=========================================================================== +2002/05/30: v0.2.2 + +* apply-config: Modified interaction with File::Find module so it + will work with older versions that do not support the -no_chdir + option. + +=========================================================================== +2002/05/24: v0.2.1 + +* mk-procmailrc: Fixed bug where setting an archive to a Period of + a year had no effect: archive would have a monthly period. + +=========================================================================== +2002/05/22: v0.2.0 + +* web-archive: Added -mharcdir option to specify location of list + specific resource files. If the file <-mharcdir>/.mrc + exists, it will be passed into mhonarc, after common.mrc. + +* web-archive: Added support for additional mhonarc options to be + specified in lists.def by defining the MHonArc-Options field. + +* web-archive: If the No-Search lists.def field is set to 1 for + a list, then searching will be disabled for the archive: no search + index will be created and the $SEARCH-FORM$ custom resource variable + will be set to the empty string. + +* web-archive: Changed some of the default pathname values of some + options. In sum, default pathnames that include "../archive/.." + had the "archive/" component removed. + +* web-archive: The following resource variables set have different + names: + + $MNAV-CGI$ => $PNAV-CGI$ + $CUR-MONTH$ => $CUR-PERIOD$ + + The old names are still set for backwards compatibility. + +* web-archive: The "[Raw]" link on the top index page of a list + archive is now "[mbox]". Also, it will show the size of the mbox + in bytes and if the mbox is gzipped. + +* lib/common.mrc.in.dist: + + . Removed options in MIMEARGS that could cause security problems. + Users can add options that fit their level of comfort. + + . Added definition of $PREV-PERIOD-LABEL$ and $NEXT-PERIOD-LABEL$ + which are defined as "Prev Period" and "Next Period", + respectively. + + These variables are used for the next/prev period links on the + index pages. Note, these values are a change from the + "Prev Month" and "Next Month". + + . The next/prev resource variables for the next/prev period links + have different names: + + $NEXT-MONTH-LINK$ => $NEXT-PERIOD-LINK$ + $PREV-MONTH-LINK$ => $PREV-PERIOD-LINK$ + +* read-mail: Added explicit "exit 0" at end of script. This is + to avoid a bogus non-zero exit status of filter-spool finds nothing + to do. + +=========================================================================== +2002/02/14: v0.1.3 + +* web-archive: Went back to calling namazu for each list folder + processed. The change in v0.1.2 defeats the ability to conserve + resources by keeping namazu from re-indexing the entire list + each time. + +* web-archive: Check status of mhonarc processing. + +* web-archive: Skip search index update if no new messages added. + +=========================================================================== +2002/02/14: v0.1.2 + +* Added new program 'mh-month-pack' that can process MH/nmh folders + and copy messages into monthly-based mailbox files. Although this + program is not used by default by MHArc archiving system, it can + be used to customize MHArc to work with MH/nmh-based setups. + +* Updated web-archive to only call namazu once on all archive + folders changed for a list instead of calling it for each archive + folder processed. Also added a cleanup check on namazu to make + sure it does not leave a stale lock around. + +=========================================================================== +2002/02/11: v0.1.1 + +* BUG FIX: Fixed regex generation bug in mk-procmailrc where ()'s + were not matched. + +=========================================================================== +2002/02/11: v0.1.0 + +First tagged release of mharc. Code imported into CVS from the RCS +files used previously. The following changes exist in this release: + +* Mailing list definition file can now be specified in lib/config.sh. + +* BUG FIX: Fixed bug in mk-procmailrc that created duplicate + recipies for lists defined with multiple addresses. + +* The Description: option in lists.def is now used as the title + for archive index pages. + +* Added No-Raw-Link: boolean option in lists.def. It can be used + to specify that a links to the raw archive data should not be + made in the main period index of a list archive. + +* Added new variable $LIST-DESC$ for .PNM.head and .PNM.foot + process for main period index generation. The variable will + expand to the Description: value in lists.def. If no + description available, it will expand to the list name. + +* read-mail will source .env.sh in same directory if it exists. + It is an easy way to sent envariables needed for the archiving + process to run properly. + +* Modified distribution bundle creation to fit in with internal + project release process. + +=========================================================================== +2002/03/08: + + Initial release. + +=========================================================================== +$Id: NEWS,v 1.48 2003/08/09 18:46:19 ehood Exp $ diff --git a/README b/README new file mode 100644 index 0000000..981c800 --- /dev/null +++ b/README @@ -0,0 +1,59 @@ +--------------------------------------------------------------------------- + README for mharc + + Copyright (C) 2002 + Earl Hood, earl@earlhood.com +____________________________________________________________________________ + + mharc is distributed under the GNU General Public License + (GPL). The rules for using and copying mharc are explained in + the file COPYING. If you cannot agree to the conditions of the + GPL but still want to use/copy/distribute the program, you must + contact the author of mharc, earl@earlhood.com, about arranging + an alternative license. + +____________________________________________________________________________ +Table of Contents + + * What is mharc? + * Installation + * Documentation + * Mailing List + * Version + * Author + +____________________________________________________________________________ +What is mharc? + + mharc is part of the MHonArc Project. mharc is a collection of + Perl scripts for generating and managing web-based searchable + mail archives. + +____________________________________________________________________________ +Installation + + See the file INSTALL, or better, the HTML version doc/install.html + for full installation and configuration instructions. + +____________________________________________________________________________ +Documentation + + Main documentation is kept in HTML format in the doc/ directory. + Load doc/index.html into your favorite HTML viewer to get started. + +--------------------------------------------------------------------------- +Mailing List + + Information about mailing lists, web archives, and other contact + information is provided in doc/contacts.html. + +____________________________________________________________________________ +Version + + The version of mharc is provided in the file VERSION. + +____________________________________________________________________________ +Author + + Earl Hood, earl@earlhood.com + $Date: 2002/09/28 05:33:08 $ diff --git a/README.FSF b/README.FSF new file mode 100644 index 0000000..9c2b556 --- /dev/null +++ b/README.FSF @@ -0,0 +1,4 @@ +customize config.sh +customize common.mrc +email address obfuscation at end of /usr/lib/MHonArc/mhinit.pl (after ------) + diff --git a/TODO b/TODO new file mode 100644 index 0000000..8dc32a0 --- /dev/null +++ b/TODO @@ -0,0 +1,18 @@ + mharc: TODO List + + => Add a POP downloading script (something like fetchmail should + probable be used). + + => Better documentation: + . Document lib/mrc/_*.mrc resources. + . More documentation on how to customize archives. + + => Remove use of Javascript in searching. + + => Replace use of make(1) with a control program written in Perl. + + => Make everything into modules and just have one command + to use to run everything (seems like too much work now). + +=========================================================================== +$Id: TODO,v 1.15 2003/08/09 18:46:19 ehood Exp $ diff --git a/VERSION b/VERSION new file mode 100644 index 0000000..f38fc53 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +0.7.3 diff --git a/bin/apply-config b/bin/apply-config new file mode 100755 index 0000000..8f97654 --- /dev/null +++ b/bin/apply-config @@ -0,0 +1,258 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: apply-config,v 1.15 2002/10/17 03:14:31 ehood Exp $ +## Description: +## Processes all '.in' template files. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::apply_config; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + +use Getopt::Long; +use File::Find; +use MHArc::Util qw(run_prg usage); + +my $clean = 0; +my $distclean = 0; +my $debug = 0; +my $noact = 0; +my $verbose = 0; + +my $clstatus = GetOptions( + "clean!" => \$clean, + "distclean!" => \$distclean, + "debug!" => \$debug, + "n!" => \$noact, + "verbose!" => \$verbose, + + "help" => \$help, + "man" => \$man +); +usage(0) unless $clstatus; +usage(1) if $help; +usage(2) if $man; + +$verbose = 1 if $debug || $noact; +$clean = 1 if $distclean; +$MHArc::Util::ECHO_CMDS = 1 if ($verbose); +$MHArc::Util::ECHO_ONLY = 1 if ($noact); + +my %done = ( ); + +if ($debug) { + $config->dump_config(\*STDERR); +} + +sub wanted { + if ($_ =~ /config\.sh$/ || + $_ =~ /config\.sh\.in$/ || + $_ =~ /config\.sh\.dist$/ || + $_ =~ /config\.sh\.in\.dist$/) { + print qq/Skipping config file "$File::Find::name"\n/ if $debug; + return; + } + if (-d $_ && ($_ =~ /RCS$/ || $_ =~ /CVS$/ || $_ =~ /SCCS$/)) { + print qq/Pruning "$File::Find::name"\n/ if $debug; + $File::Find::prune = 1; + return; + } + if (-d $_ || !/\.in(?:\.dist)?$/o) { + print qq/Ignoring "$File::Find::name"\n/ if $debug; + return; + } + if ($done{$File::Find::name}) { + print qq/Skipping "$File::Find::name", already processed.\n/ + if $debug; + return; + } + + my $file = $_; + print qq/Checking "$File::Find::name"...\n/ if $debug; + if ($file =~ s/\.dist$//) { + if (!$clean) { + if (! -e $file) { + run_prg('/bin/cp', $_, $file); + run_prg('/bin/chmod', 'u+w', $file); + } + } + $done{$File::Find::name} = 1; + $done{"$File::Find::dir/$file"} = 1; + } + + my $file_out = $file; + $file_out =~ s/\.in$//; + + if ($clean) { + run_prg('/bin/rm', $file_out) if (-e $file_out); + run_prg('/bin/rm', $file) if ($distclean && (-e $file) && + (-e "$file.dist")); + return; + } + + print qq|Processing "$File::Find::dir/$file"\n| if $verbose; + if (!$noact) { + local(*IN, *OUT); + open(IN, $file) || + die qq|ERROR: Unable to open "$File::Find::dir/$file": $!\n|; + open(OUT, ">$file_out") || + die qq|ERROR: Unable to create "$File::Find::dir/$file_out": $!\n|; + + my($line); + while (defined($line = )) { + $line =~ s/\@\@([^@]*)\@\@/$config->{$1}/g; + print OUT $line; + } + close(IN); + close(OUT); + if (-x $file) { + run_prg('/bin/chmod', 'a+x,go-w', $file_out); + } + } +} + +if (!@ARGV) { + my $home = $config->{'SW_ROOT'}; + $home = "$Dir/.." unless $home; + push(@ARGV, $home); + + my($dir); + foreach $dir ($config->{'HTML_DIR'}, + $config->{'CGI_DIR'}, + $config->{'MBOX_DIR'}, + $config->{'INFO_DIR'}, + $config->{'MKNMZ_TMPL_DIR'}) { + + next unless $dir =~ /\S/; + if (! -e $dir) { + warn qq/Warning: "$dir" does not exist\n/; + next; + } + $dir =~ s/\/+$//; + if ($dir !~ /^\Q$home\E\//o) { + push(@ARGV, $dir); + } + } +} + +find(\&wanted, @ARGV); + +##--------------------------------------------------------------------------## +__END__ + +=head1 NAME + +apply-config - Process input template files based upon configuration settings. + +=head1 SYNOPSIS + + apply-config [options] [dir ...] + +=head1 DESCRIPTION + +This program processes input template files and expands variables +referenced to values specified in Cmharc-rootE/lib/config.sh>. + +Template files are designated by the C<.in> filename extentions. For a +given file, if C.in.dist> exists and C.in> does not, +C.in.dist> will be copied to C.in> before processing. + +Variable references in template files are denoted as follows: + + @@VARIABLE_NAME@@ + +If the specified variable name is defined, the reference will be +replaced with the empty string. + +=head1 OPTIONS + +Any non-option arguments are treated as directories to recursively +scan for template files. If no directories are specified, then +C<$SW_ROOT> is used as defined in Cmharc-rootE/lib/config.sh>. + +=over + +=item -clean + +Remove all files that have a C<.in> version. This option is useful +to clean up all files generated from templates. + +=item -distclean + +Remove all files that have a C<.in> version and remove all C<.in> +files that have a C<.in.dist> version. This option is useful to +clean up all files for generating a distribution bundle. + +B Use this option with care since it will delete all +customized versions of C<.in> templates. This option is mainly +for use by mharc developers. + +=item -debug + +Print out alot of information on what is going on. This options +prints out more information than C<-verbose>. + +=item -n + +Just echo what would be done, but do not do it. + +=item -verbose + +Echo out status on any operation that modifies files. + +=back + +=head1 FILES + +=over + +=item Cmharc-rootE/lib/config.sh> + +Configuration file defining variables values. + +=back + +=head1 VERSION + +C<$Id: apply-config,v 1.15 2002/10/17 03:14:31 ehood Exp $> + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mhArc +distribution. + +=cut + diff --git a/bin/compress-files b/bin/compress-files new file mode 100755 index 0000000..3edcfeb --- /dev/null +++ b/bin/compress-files @@ -0,0 +1,180 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: compress-files,v 1.8 2002/09/15 03:33:08 ehood Exp $ +## Description: +## Gzip files matching a specified pattern and older then specified +## time period. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::compress_files; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use File::Find; +use MHArc::Util qw( usage ); + +# Amount of seconds in a year +my $year_secs = 31536000; + +my %opt = ( ); +my $clstatus = GetOptions(\%opt, + "debug!", + "pattern=s", + "mbox-mode!", + "mtime=i", + "n!", + + 'help', + 'man' +); +usage(0) unless $clstatus; +usage(1) if $opt{'help'}; +usage(2) if $opt{'man'}; + +my $debug = $opt{'debug'}; +my $mbox_mode = $opt{'mbox-mode'}; +my $pattern = $opt{'pattern'} || ($mbox_mode ? '^\d+(?:-\d+)?$' : '^[^.]'); +my $compress_time = $opt{'mtime'} || 31; +my $noact = $opt{'n'}; + +my $time = time; +$compress_time *= 24 * 3600; # convert to seconds + +sub wanted { + if (-d $_ || + /\.gz$/i || + !/$pattern/o) { + print qq/Ignoring "$File::Find::name"\n/ if $debug; + return; + } + + my $mtime = (stat($_))[9]; + print qq/$File::Find::name (/, scalar(localtime($mtime)), qq/)\n/ if $debug; + + if ($mbox_mode && (/^\d+$/)) { + if ($time-$mtime < ($compress_time+$year_secs)) { + print qq/Skipping "$File::Find::name"\n/ if $debug; + return; + } + } elsif ($time-$mtime < $compress_time) { + print qq/Skipping "$File::Find::name"\n/ if $debug; + return; + } + + print qq/Compressing "$File::Find::name"\n/ if $debug; + if ($noact) { + print qq/gzip $File::Find::name\n/; + } else { + if (system('gzip', $_)) { + die qq/gzip $File::Find::name failed: $?\n/; + } + } +} + +if ($#ARGV < 0) { + die qq/No directories, or files, specified\n/; +} +find(\&wanted, @ARGV); + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +compress-files - Gzip files not modified over a given period of time + +=head1 SYNOPSIS + + compress-files [options] [ ...] + +=head1 DESCRIPTION + +This program is part of mharc. This program is used to compress +files that have not been modified over a given period of time. + +=head1 OPTIONS + +=over + +=item C<-debug> + +Print out alot of status information. + +=item C<-help> + +Print out usage information. + +=item C<-n> + +Print the commands that would be executed, but do not execute them. + +=item C<-mbox-mode> + +A hack for compress mailbox files. When this option is specified, +the default C<-pattern> is C<^\d+(?:-\d+)?$>. This pattern basically +searches for files with names in YYYY and YYYY-MM format. Files with +names in YYYY format are handled slightly different. When checking +if the file can be compressed, a year is added to C<-mtime>. + +=item C<-man> + +Print out manpage. + +=item C<-mtime> I + +Modification time in days a file has to be older than to get compressed. +If this option is not specified, 31 days is used. + +=item C<-pattern> I + +Perl regular expression that represents files that should be +checked. If not specifed, the following regex is used: +C<^[^.]>. + +=back + +=head1 SEE ALSO + +L + +=head1 VERSION + +$Id: compress-files,v 1.8 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/compress-mboxes b/bin/compress-mboxes new file mode 100755 index 0000000..73a88ff --- /dev/null +++ b/bin/compress-mboxes @@ -0,0 +1,179 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: compress-mboxes,v 1.8 2002/09/15 03:33:08 ehood Exp $ +## Description: +## Script to compress older mailbox inorder to save disk space. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::compress_mboxes; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( exec_prg usage ); + +MAIN: { + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + "debug!", + "n!", + + 'help', + 'man' + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + my $debug = $opt{'debug'}; + my $noact = $opt{'n'}; + + $MHArc::Util::ECHO_CMDS = $noact || $debug; + $MHArc::Util::ECHO_ONLY = $noact; + + $ENV{'PATH'} = join(':', join('/', $config->{'SW_ROOT'}, 'bin'), + '/usr/local/bin', + '/bin', + '/usr/bin'); + + # Sanity check on $MBOX_DIR + if (! -d $config->{'MBOX_DIR'}) { + die qq/ERROR: "/, $config->{'MBOX_DIR'}, qq/" is not a directory!/; + } + + # Call compress-files + my @cmd_args = ( + '-mbox-mode', + ); + if ($noact) { + push(@cmd_args, '-n'); + } + if ($debug) { + push(@cmd_args, '-debug'); + } + exec("$Dir/compress-files", @cmd_args, $config->{'MBOX_DIR'}); + die qq/exec @_ failed: $?\n/; + +} # End: MAIN + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +compress-mboxes - Gzip old raw mailbox files + +=head1 SYNOPSIS + + compress-mboxes + compress-mboxes [options] + +=head1 DESCRIPTION + +This program is part of mharc and has the responsibility of gzipping +mailbox files that have not been modified in over a month to save +disk space. + +Generally, this program will be invoked automatically from C +on a periodic basis. + +This program will use the C variable from +Cmharc-rootE/lib/config.sh> as the root directory to search for +mailbox files. Any file matching the following regex, + + ^\d+(?:-\d+)?$ + +is considered to be a mailbox file. + +=head1 OPTIONS + +=over + +=item C<-debug> + +Print out alot of status information. + +=item C<-help> + +Print out usage information. + +=item C<-n> + +Print the commands that would be executed, but do not execute them. + +=item C<-man> + +Print out manpage. + +=back + +=head1 FILES + +=over + +=item Cmharc-rootE/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 NOTES + +=over + +=item * + +This program basically invokes L with the proper +arguments to do the actual work. + +=back + +=head1 SEE ALSO + +L + +=head1 VERSION + +$Id: compress-mboxes,v 1.8 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/config-check b/bin/config-check new file mode 100755 index 0000000..050a888 --- /dev/null +++ b/bin/config-check @@ -0,0 +1,87 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: config-check,v 1.1 2002/09/20 02:56:31 ehood Exp $ +## Description: +## Dump mharc config to stdout. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::config_dump; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## + +use MHArc::Config; + +MAIN: { + my $config = load_config(); + $config->dump_config(\*STDOUT); +} + +sub load_config { + MHArc::Config->load("$Dir/../lib/config.sh"); +} + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +config-check - Check mharc configuration file + +=head1 SYNOPSIS + + config-check + +=head1 DESCRIPTION + +This programs prints out the expanded variable values defined +in C to standard output. + +=head1 FILES + +=over + +=item C/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 VERSION + +$Id: config-check,v 1.1 2002/09/20 02:56:31 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/ed b/bin/ed new file mode 120000 index 0000000..5585639 --- /dev/null +++ b/bin/ed @@ -0,0 +1 @@ +/home/list/bin/ed \ No newline at end of file diff --git a/bin/extract-mesg-date b/bin/extract-mesg-date new file mode 100755 index 0000000..4fa13c7 --- /dev/null +++ b/bin/extract-mesg-date @@ -0,0 +1,188 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: extract-mesg-date,v 1.4 2002/09/15 03:33:08 ehood Exp $ +## Description: +## See POD below or run program with -man option. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::extract_mesg_date; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( usage ); +use MHArc::MailUtil qw( extract_date ); + +require 'mhamain.pl'; + +my $debug = 0; +my $verbose = 0; +my $time_fmt = '%Y-%m'; + +MAIN: { + # Load mhonarc code + mhonarc::initialize(); + mhonarc::open_archive( + '-noarg', + '-quiet', + '-posixstrftime' + ) || die qq/ERROR: Unable to load MHonArc library\n/; + mhonarc::close_archive(); + + # Grap command-line options + my($opt_dfs); + my $clstatus = GetOptions( + "debug!" => \$debug, + "datefields=s" => \$opt_dfs, + "fmt=s" => \$time_fmt, + + "help" => \$help, + "man" => \$man + ); + usage(0) unless $clstatus; + usage(1) if $help; + usage(2) if $man; + + if ($debug) { + $MHArc::MailUtil::Debug = 1; + } + + my @date_fields = (); + if (defined($opt_dfs)) { + @date_fields = split(/:/, $opt_dfs); + } elsif (defined($config->{'MSG_DATE_FIELDS'})) { + @date_fields = split(/:/, $config->{'MSG_DATE_FIELDS'}); + } + print "date_fields=@date_fields\n" if $debug; + my($fields, $header) = readmail::MAILread_file_header(\*STDIN); + print mhonarc::time2str( + $time_fmt, extract_date($fields, @date_fields), 1); + +} # End: MAIN + +##--------------------------------------------------------------------------## +__END__ + +=head1 NAME + +extract-mesg-date - Retrieve date of a mail message + +=head1 SYNOPSIS + + extract-mesg-date [options] + +=head1 DESCRIPTION + +This program extracts the date of a mail message read in from +standard input. The date of the message is determined by +examining the following mail header fields in order: +C, C, C. The fields checked +can be changed with the C<-datefields> option. If no date +is found, than current local time is used. + +The date of the message will be echoed to standard output. +The format of the date is controled by the C<-fmt> option. + +This program is provided as part of mharc to provide the ability to +to determine the dates of messages during filtering. +Example shell command usage: + + mesg_date=`cat message | extract-mesg-date` + +Example useage within a procmail recipe: + + :0 Wi + MESGDATE_=| extract-mesg-date -fmt '%Y-%m' + + :0: + $MBOXROOT/.listsadmin/$MESGDATE_ + +=head1 OPTIONS + +=over + +=item C<-datefields> I + +Specifies the message header fields to examine in determining the +date of the message. Field names are separated by a colon. +For example, + + -datefields x-archive-date:received:date + +tells that C, C, and C should +be examined. + +=item C<-debug> + +Print out debugging information. + +B + +=item C<-fmt> I + +The time format to use. The format string syntax is the same as +defined by C. + +If C<-fmt> is not specified, than "C<%Y-%m>" is used. + +=item C<-help> + +Print out help message. + +=item C<-man> + +Print out the manpage. + +=back + +=head1 DEPENDENCIES + +This program uses functions within MHonArc's library. Therefore, +MHonArc must be installed on your system and the MHonArc libraries +located within Perl's include path. + +=head1 VERSION + +$Id: extract-mesg-date,v 1.4 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/filter-spool b/bin/filter-spool new file mode 100755 index 0000000..2533866 --- /dev/null +++ b/bin/filter-spool @@ -0,0 +1,309 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: filter-spool,v 1.11 2002/09/27 05:01:07 ehood Exp $ +## Description: +## Script to grab mail spool and filter mail to raw mbox archives. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::filter_spool; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( cmd run_prg usage ); + +my @_term_sigs = qw( + ABRT ALRM BUS FPE HUP ILL INT IOT PIPE POLL PROF QUIT SEGV + TERM TRAP USR1 USR2 VTALRM XCPU XFSZ +); + +my $Procmail = $config->{'PROCMAIL'} || 'procmail'; +my $Lockfile = $config->{'LOCKFILE'} || 'lockfile'; +my $Formail = $config->{'FORMAIL'} || 'formail'; + +my $TmpSpool = '.newmail'; +my $TmpSpoolLock = $TmpSpool.'.lock'; + +MAIN: { + # Make sure umask is set to make things readable by default + umask 022; + + # Grap command-line options + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + 'verbose!', + 'home=s', + 'html-dir=s', + 'is-spool!', + 'log-dir=s', + 'lock-timeout=i', + 'mail=s', + 'mbox-dir=s', + 'procmailrc=s', + 'procmailvars=s', + + 'help', + 'man', + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + my $verbose = $opt{'verbose'}; + if ($verbose) { + $MHArc::Util::ECHO_CMDS = 1; + } + + my $home = $opt{'home'} || + $config->{'SW_ROOT'} || + "$Dir/.."; + my $html_dir = $opt{'html-dir'} || + $config->{'HTML_DIR'} || + join('/', $home, 'html'); + my $log_dir = $opt{'log-dir'} || + $config->{'LOG_DIR'} || + join('/', $home, 'log'); + my $mbox_dir = $opt{'mbox-dir'} || + $config->{'MBOX_DIR'} || + join('/', $home, 'mbox'); + my $procmailrc = $opt{'procmailrc'} || + $config->{'PROCMAILRC'} || + join('/', $home, 'procmailrc.mharc'); + my $procmailvars = $opt{'procmailvars'} || + $config->{'PROCMAILVARS'} || + ""; + my $mail = $opt{'mail'} || + $config->{'ORGMAIL'} || + join('/', '/var/mail', $ENV{'LOGNAME'}); + my $is_spool = $opt{'is-spool'} || + $config->{'IS_MAIL_SPOOL'} || + 1; + my $lock_to = $opt{'lock-timeout'} || + $config->{'ORGMAIL_LOCK_TIMEOUT'} || + 3600; + + die qq/ERROR: "$home" not a directory/ + if (! -d $home); + die qq/ERROR: Unable to change directory to "$home": $!/ + unless chdir($home); + + # Make sure certain directories exist + cmd('mkdir', '-p', $log_dir, $mbox_dir, $html_dir); + + # Check that we have data to process + my $have_spool = 1; + if ((! -e $mail) || ((stat($mail))[7] == 0)) { + $have_spool = 0; + if (! -e $TmpSpool) { + print qq/"$mail" does not exist or is zero bytes/ if $verbose; + exit 1; + } + print qq/No new mail, but $TmpSpool exists\n/ if $verbose; + } + + if (cmd("$Lockfile -r5 -l$lock_to $TmpSpoolLock 2>/dev/null") != 0) { + print qq/Unable to obtain lock, exiting/ if $verbose; + exit 1; + } + + eval { + local @SIG{@_term_sigs} = (\&clean_lock) x scalar(@_term_sigs); + if ($have_spool) { + if ($is_spool) { + run_prg($Lockfile, "-l$lock_to", '-ml'); + } else { + run_prg($Lockfile, "-l$lock_to", "$mail.lock"); + } + if (cmd("/bin/cat '$mail' >>$TmpSpool") == 0) { + cmd("/bin/cat /dev/null >'$mail'"); + } + if ($is_spool) { + cmd($Lockfile, '-mu'); + } else { + unlink("$mail.lock") || + warn qq/Warning: Unable to remove "$mail.lock": $!\n/; + } + } + + if (cmd("$Formail -s $Procmail $procmailrc $procmailvars <$TmpSpool") + == 0) { + unlink($TmpSpool) || + warn qq/Warning: Unable to remove "$TmpSpool"\n/; + } + }; + clean_lock(); + if ($@) { + die $@, "\n"; + } + +} # End: MAIN + +##---------------------------------------------------------------------------## + +sub clean_lock { + unlink($TmpSpoolLock); +} + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +filter-spool - Filter incoming mail into raw archives + +=head1 SYNOPSIS + + filter-spool + filter-spool [options] + +=head1 DESCRIPTION + +This program is part of mharc and has the responsibility of filtering +incoming mail into the raw message archives. This script is called +by the L script before L +is invoked. + +=head1 OPTIONS + +This program is generally called without any command-line options +since it will read Cmharc-rootE/lib/config.sh> for all configurable +options. However, the following command-line options are +available: + +=over + +=item C<-help> + +Print out usage information. + +=item C<-home> I + +Root pathname of archiving software and data. If not specified, +C variable in C is used, else the parent directory +that contains this program is used. + +=item C<-html-dir> I + +Root pathname containing HTML archives. If not specified, +C variable in C is used, else C/html> +is used. + +B This program does not do any processing of the HTML archives. +This option is used to insure that the HTML archive root directory +exists for subsequent processing by other mharc scripts. + +=item C<-is-spool> + +Specifies that C<-mail> represents a mail spool file. If not +specified, the value of the C variable in C +is used, else C<-mail> is assumed to be a mail spool file. + +=item C<-lock-timeout> I + +The age of a lock before it is forceably removed. This is used +to help deal with stale locks. + +If this option is not specified, C variable in +C is used, else C<3600> is used. + +=item C<-log-dir> I + +Root pathname to place log files. If not specified, +C variable in C is used, else C/log> +is used. + +=item C<-mail> I + +Pathname to incoming mailbox file. +If not specified, C variable in C is used, +else C is used, where C<$LOGNAME> represents the +value of the C environment variable. + +=item C<-man> + +Print out entire manpage. + +=item C<-mbox-dir> I + +Root pathname containing raw mailbox archives. If not specified, +C variable in C is used, else C/mbox> +is used. + +=item C<-procmailrc> I + +Pathname to procmailrc file to use when filtering mail. +If not specified, C variable in C is used, +else C/procmailrc.mharc> is used. + +=item C<-procmailvars> I + +Additional variables to pass into C. +If not specified, C variable in C is used. + +=item C<-verbose> + +Print out status messages. + +=back + +=head1 EXIT VALUES + +If there was mail to process, and no errors occurred during processing, +a zero exit status will be returned. Otherwise, a non-zero exit status +will be returned. + +=head1 FILES + +=over + +=item Cmharc-rootE/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 VERSION + +$Id: filter-spool,v 1.11 2002/09/27 05:01:07 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/gc-search-indexes b/bin/gc-search-indexes new file mode 100755 index 0000000..5f6e306 --- /dev/null +++ b/bin/gc-search-indexes @@ -0,0 +1,192 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: gc-search-indexes,v 1.6 2002/09/15 03:33:08 ehood Exp $ +## Description: +## Script to do garbage collection on Namazu search indexes. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::compress_mboxes; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( cmd usage ); + +MAIN: { + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + "debug!", + "n!", + + 'help', + 'man' + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + my $debug = $opt{'debug'}; + my $noact = $opt{'n'}; + + $ENV{'PATH'} = join(':', join('/', $config->{'SW_ROOT'}, 'bin'), + '/usr/local/bin', + '/bin', + '/usr/bin'); + + $MHArc::Util::ECHO_CMDS = $noact || $debug; + $MHArc::Util::ECHO_ONLY = $noact; + + print "PATH=$ENV{'PATH'}\n" if $debug; + + # Sanity check on $HTML_DIR + my $html_dir = $config->{'HTML_DIR'}; + if (! -d $html_dir) { + die qq/ERROR: "/, $html_dir, qq/" is not a directory!/; + } + print qq/HTML_DIR=$html_dir\n/ if $debug; + + # Get list of possible directories + opendir(HTML_DIR, $html_dir) || + die qq/ERROR: Unable to open "$html_dir" for reading: $!\n/; + my @dirs = map { "$html_dir/$_" } + grep { ($_ ne '.') && + ($_ ne '..') && + (-d "$html_dir/$_") && + (-e "$html_dir/$_/NMZ.t") } readdir(HTML_DIR); + + # Check that we have something to do + if (!scalar(@dirs)) { + print "No search indexes found\n" if $debug; + exit 0; + } + print "Searchable directories:\n\t", + join("\n\t", @dirs), "\n" if $debug; + + # Run garbage collection + my @cmd_args = ( '-b' ); + if ($debug) { + push(@cmd_args, '-v'); + } else { + push(@cmd_args, '-q'); + } + foreach (@dirs) { + if (cmd('gcnmz', @cmd_args, $_) != 0) { + warn qq/Warning: Command "gcnmz @cmd_args $_" failed: $?\n/; + } + } + +} # End: MAIN + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +gc-search-indexes - Garbage collect archive search indexes. + +=head1 SYNOPSIS + + gc-search-indexes + gc-search-indexes [options] + +=head1 DESCRIPTION + +This program is part of mharc, and has the responsibility of performing +garbage collection on archive search indexes. + +Generally, this program will be invoked automatically from C +on a periodic basis. + +This program will use the C variable from +Cmharc-rootE/lib/config.sh> as the root directory to search for +searchable archives. + +=head1 OPTIONS + +=over + +=item C<-debug> + +Print out alot of information on what is going on. + +=item C<-help> + +Print out usage information. + +=item C<-n> + +Print the commands that would be executed, but do not execute them. + +=item C<-man> + +Print out manpage. + +=back + +=head1 FILES + +=over + +=item Cmharc-rootE/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 NOTES + +=over + +=item * + +This program should be invoked occasionally, like once a week. The +template crontab provided with mharc provides a useful crontab entry +for invoking this program. + +=back + +=head1 VERSION + +$Id: gc-search-indexes,v 1.6 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/logcmd b/bin/logcmd new file mode 100755 index 0000000..81e7427 --- /dev/null +++ b/bin/logcmd @@ -0,0 +1,262 @@ +#!/usr/bin/perl -w +##------------------------------------------------------------------------## +## File: +## $Id: logcmd,v 1.1 2002/10/01 22:49:46 ehood Exp $ +## Author: +## Earl Hood earl@earlhood.com +## Description: +## Program to log the output of a program. +## POD at __END__. +##------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood, earl@earlhood.com +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##------------------------------------------------------------------------## + +package Devtools::locmd; + +use strict; +use Getopt::Long; +use POSIX; + +my $our_name = 'logcmd'; +my @cmd = ( ); +my $cmd_name = ""; +my $time_fmt = '%Y-%m-%d %H:%M:%S'; + +my $first_time = 1; +my $log_file = '-'; +my $log_fh; +my $child_pid; + +MAIN: { + my @mailto = ( ); + my ($help, $man); + + GetOptions( + 'log=s' => \$log_file, + 'mailto=s@' => \@mailto, # not implemented, yet + 'time-fmt=s' => \$time_fmt, + + 'help' => \$help, + 'man' => \$man, + ) || die qq/ERROR: Use -help for usage information\n/; + usage(1, 0) if ($help); + usage(2, 0) if ($man); + + if (!@ARGV) { + die qq/ERROR: No command specified\n/; + } + ($cmd_name = $ARGV[0]) =~ s/.*\///; + @cmd = @ARGV; + + $child_pid = open(CMD, "-|"); + if ($child_pid) { # parent + { + local @SIG{'PIPE','TERM','INT'} = (sub { + log_line($our_name, $$, "Caught signal: SIG$_[0]\n"); + }) x 3; + + while () { + log_line($cmd_name, $child_pid, $_); + } + } + if (!close(CMD)) { + log_line($our_name, $$, qq/Non-zero exit for [$child_pid]: $?\n/); + } + log_close(); + + } else { # child + open(STDERR, '>&STDOUT'); + exec(@cmd) || die qq/ERROR: Cannot exec "@cmd": $!\n/; + } + +} # End: MAIN + +##=========================================================================## + +sub format_time { + POSIX::strftime($time_fmt, localtime); +} + +sub log_line { + my $label = shift; + my $pid = shift; + my $line = shift; + my $have_newline = $line =~ /\n\Z/; + my $eol = ($have_newline) ? "" : "\n"; + if ($first_time) { + $first_time = 0; + log_open($log_file); + print $log_fh join('', '[',format_time(),']',' [',$$,'] ',$our_name,': ', + "Command: [$child_pid] @cmd\n"); + } + print $log_fh join('', '[',format_time(),']',' [',$pid,'] ',$label,': ', + $line,$eol); +} + +sub log_open { + my $file = shift; + if (!defined($file) || $file eq '-') { + $log_fh = \*STDOUT; + } else { + open(LOG, ">>$file") || + die qq/ERROR: Unable to open "$file" for appending: $!\n/; + $log_fh = \*LOG; + } +} + +sub log_close { + if (!$first_time) { + if ($log_fh != \*STDOUT) { + close($log_fh) || + warn qq/Warning: Problem closing log file: $!\n/; + } + } +} + +sub usage { + require Pod::Usage; + my $verbose = shift || 0; + my $exit_code = shift; + + if ($verbose == 0) { + Pod::Usage::pod2usage(-verbose => $verbose); + } else { + my $pager = $ENV{'PAGER'} || 'more'; + local(*PAGER); + my $fh = (-t STDOUT && open(PAGER, "|$pager")) ? \*PAGER : \*STDOUT; + Pod::Usage::pod2usage(-verbose => $verbose, + -output => $fh); + close(PAGER) if ($fh == \*PAGER); + } + defined($exit_code) && exit($exit_code); +} + +##=========================================================================## +__END__ + +=head1 NAME + +logcmd - Log the output of a program. + +=head1 SYNOPSIS + + logcmd [options] -- [command-args ...] + +=head1 DESCRIPTION + +B logs the output of a program. This program is useful +for logging program output for programs that do not have a built-in +logging facility. + +Typical usage is to use B in crontabs to log any command +output to a file instead of mail automatically being sent to the +crontab owner. + +When invoking B, you B use "C<-->" to separate the +B and its options from the command you are invoking. For +example, + + logcmd -log out.log -- some-cmd arg1 arg2 arg3 + +This tells B to run "C" and have +any output goto C. + +Each line logged will be preceded by a timestamp, the process ID, and +the name of the command. Also, before the first log message for a command +is printed, B will print out a line giving the full command-line +of the command invoked. Example: + + [2002-07-30 14:52:13] [25392] logcmd: Command: [25393] mhonarc -quiet -out out /var/archiver/mail + [2002-07-30 14:52:13] [25393] mhonarc: + [2002-07-30 14:52:13] [25393] mhonarc: Warning: Unrecognized character set: utf-8 + [2002-07-30 14:52:13] [25393] mhonarc: Message-Id: <1022714308.2028.1.camel@xxx.xxx.xxx> + [2002-07-30 14:52:13] [25393] mhonarc: Message Number: 00367 + [2002-07-30 14:52:13] [25393] mhonarc: + [2002-07-30 14:52:13] [25393] mhonarc: Warning: Unrecognized character set: utf-8 + [2002-07-30 14:52:13] [25393] mhonarc: Message-Id: <1022716087.2028.3.camel@xxx.xxx.xxx> + [2002-07-30 14:52:13] [25393] mhonarc: Message Number: 00368 + [2002-07-30 14:52:14] [25393] mhonarc: + [2002-07-30 14:52:14] [25393] mhonarc: Warning: Unrecognized character set: utf-8 + [2002-07-30 14:52:14] [25393] mhonarc: Message-Id: <1024365245.9394.0.camel@xxx.xxx.xxx> + [2002-07-30 14:52:14] [25393] mhonarc: Message Number: 00398 + + +=head1 OPTIONS + +=over + +=item C<-help> + +Display SYNOPSIS and OPTIONS sections. + +=item C<-log> I + +Command output will be logged to I. If this option is +not specified, then standard out is used. + +=item C<-man> + +Display manpage. + +=item C<-time-fmt> I + +Date/time format to use when printing time before each logged line. +The format string should be in C(3) format. If this +option is not specified, then "C<%Y-%m-%d %H:%M:%S>" is used. + +=item C<--> + +Terminate option processing for B. Any arguments after +C<--> comprise the command that B should execute. + +=back + +=head1 EXAMPLES + +The following is an example crontab entries of a user that has +any output of commands goto a personal log file: + + 0 2 * * * logcmd -log $HOME/log/cron.log -- $HOME/bin/clean_account + 0 3 * * * logcmd -log $HOME/log/cron.log -- $HOME/bin/pack_mail + +=head1 LIMITATIONS + +=over + +=item * + +B performs no file locking on the log file specified. To help +minimize that a single output line to the log does not get broken up +due to multiple writers, the entire line is dumped in a single call +to Perl's C operator. + +=back + +=head1 DEPENDENCIES + +C, C. + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program comes with ABSOLUTELY NO WARRANTY and may be copied only +under the terms of the GNU General Public License. + +=cut + diff --git a/bin/mbox-month-pack b/bin/mbox-month-pack new file mode 100755 index 0000000..63021e3 --- /dev/null +++ b/bin/mbox-month-pack @@ -0,0 +1,322 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: mbox-month-pack,v 1.4 2002/09/15 03:33:08 ehood Exp $ +## Description: +## See POD below or run program with -man option. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::mbox_month_pack; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( usage ); + +# For MHonArc date/time utilities and message head parsing +require 'mhamain.pl'; + +my $debug = 0; +my $verbose = 0; +my $noop = 0; +my $noanno = 0; +my $all = 0; + +my $outdir = '.'; +my $yearly = 0; + +my $msgsep = '^From '; + +MAIN: { + # Load mhonarc code + mhonarc::initialize(); + mhonarc::open_archive('-noarg', '-quiet') || + die qq/ERROR: Unable to load MHonArc library\n/; + mhonarc::close_archive(); + + # Grap command-line options + my $clstatus = GetOptions( + "debug!" => \$debug, + "msgsep=s" => \$msgsep, + "outdir=s" => \$outdir, + "verbose!" => \$verbose, + "yearly!" => \$yearly, + + "help" => \$help, + "man" => \$man + ); + usage(0) unless $clstatus; + usage(1) if $help; + usage(2) if $man; + + $verbose = 1 if $noop; + $verbose = 1 if $debug; + + push(@ARGV, '-') if (!@ARGV); + + if ($verbose) { + select(STDOUT); $| = 1; + } + + local(*MBOX, $_); + my($fh, $sep, $header, $fields, $body, $mon, $yr); + my @date; + + MBOX: foreach $mbox (@ARGV) { + print qq/Processing mbox "$mbox"/ if $verbose; + + if ($mbox eq '-') { + $fh = \*STDIN; + } else { + if (!open(MBOX, $mbox)) { + warn qq/Warning: Unable to open "$mbox": $!\n/; + next MBOX; + } + $fh = \*MBOX; + } + + print qq/Debug: Scanning for first separator/ if $debug; + $sep = undef; + while (<$fh>) { + if (/$msgsep/o) { + $sep = $_; + last; + } + } + + while (defined($sep)) { + print '.' if $verbose && !$debug; + + # Grab message header and date. + ($fields, $header) = read_mail_header($fh); + #dump_header(\*STDOUT, $fields) if $debug; + + print qq/Debug: separator=$sep/ if $debug; + if ($use_sep_date) { + @date = mhonarc::parse_date($sep); + } else { + @date = ( ); + } + + + if (!@date) { + if (defined($fields->{'received'})) { + my @ra = split(/;/, $fields->{'received'}[0]); + print qq/Debug: Received date=$ra[-1]\n/ if $debug; + @date = mhonarc::parse_date(pop(@ra)); + } elsif (defined($fields->{'date'})) { + @date = mhonarc::parse_date($fields->{'date'}[0]); + } + } + + print qq/Debug: \@date=/, join('|',@date), qq/\n/ if $debug; + if (@date) { + ($mon, $yr) = + (localtime(mhonarc::get_time_from_date(@date[1..$#date])))[4,5]; + ++$mon; + $yr += 1900; + + } else { + warn qq/Warning: No date found for message, using current\n/, + qq/ Message-Id: /, $fields->{'message-id'}[0], qq/\n/, + qq/ Subject: /, $fields->{'subject'}[0], qq/\n/; + ($mon, $yr) = (localtime(time))[4,5]; + ++$mon; + $yr += 1900; + } + print qq/Debug: year=$yr, month=$mon\n/ if $debug; + + $sep = dump_to_mbox($fh, $yr, $mon, $sep, $header); + + } + } continue { + print "\n" if $verbose; + } + + +} # End: MAIN + +##--------------------------------------------------------------------------## + +sub read_mail_header { + readmail::MAILread_file_header(@_); +} + +sub dump_to_mbox { + my $fh = shift; + my $yr = shift; + my $mon = shift; + # rest of arguments comprise the header + + my $out_file = join('/', $outdir, + ($yearly ? sprintf("%04d", $yr) : + sprintf("%04d-%02d", $yr, $mon))); + + local(*OUT); + open(OUT, ">>$out_file") || + die qq/ERROR: Unable to open "$out_file": $!\n/; + print qq/Debug: Appending to "$out_file"\n/ if $debug; + + # Print separator/header + print OUT @_, "\n"; + + # Get body + my $body = ''; + my $sep = undef; + local $_; + while (<$fh>) { + if (/$msgsep/o) { + $sep = $_; + last; + } + print OUT $_; + } + $sep; +} + +sub dump_header { + my $fh = shift; + my $fields = shift; + my($key, $a, $value); + foreach $key (sort keys %$fields) { + $a = $fields->{$key}; + if (ref($a)) { + foreach $value (@$a) { + print $fh "$key: $value\n"; + } + } else { + print $fh "$key: $a\n"; + } + } +} + + +##--------------------------------------------------------------------------## +__END__ + +=head1 NAME + +mbox-month-pack - Copy mailbox messages into monthly mailbox files. + +=head1 SYNOPSIS + + mbox-month-pack [options] folder ... + +=head1 DESCRIPTION + +This program copies mailbox messages into monthly (or yearly if +the C<-yearly> option is specified) mailbox files. By default, +monthly mailbox files are created with filenames of I format. +If the C<-yearly> option is specified, than messages will be split +into yearly, I, files. If a mailbox file already exists, +messages will be appended to it. + +This program is provided as part of mharc to provide the ability to +import existing mailbox messages into mharc archives, or as a possible +replacement for L for sites that have +alternate methods for managing incoming mail. + +=head1 OPTIONS + +=over + +=item C<-debug> + +Like C<-verbose>, but prints much more. + +=item C<-help> + +Print out help message. + +=item C<-man> + +Print out the manpage. + +=item C<-outdir> I + +Directory to place mailbox files. If not specified, the +current working directory is used. + +=item C<-verbose> + +Print status of what is going on. + +=item C<-yearly> + +Generate yearly-based mailbox files instead of monthly-based. + +=back + +=head1 DEPENDENCIES + +This program uses MHonArc's date parsing functions. Therefore, +MHonArc must be installed on your system and the MHonArc libraries +located within Perl's include path. + +=head1 LIMITATIONS + +=over + +=item * + +This program does not remember what messages it has processed. +For example, if you run the program twice in a row like the following: + + prompt> mbox-month-pack mail.mbx + prompt> mbox-month-pack mail.mbx + +The resulting monthly mailbox files will contain two of each message. + +=item * + +Appending to pre-existing gzipped monthly, and yearly, mailbox files +are not recognized when splitting input. If you want output to be +appended to existing compressed mailboxes, you must uncompress them +first before invoking this program. + +=back + +=head1 VERSION + +$Id: mbox-month-pack,v 1.4 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/mh-month-pack b/bin/mh-month-pack new file mode 100755 index 0000000..a06a542 --- /dev/null +++ b/bin/mh-month-pack @@ -0,0 +1,285 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: mh-month-pack,v 1.5 2002/10/11 23:52:28 ehood Exp $ +## Description: +## See POD below or run program with -man option. +##--------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::mh_month_pack; + +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } + +use Getopt::Long; + +use constant SCAN_FORMAT_STR => + '%(msg) %<{x-mharc-packed}*%>%(year{date})-%02(mon{date})-%02(mday{date})'; + +my $debug = 0; +my $verbose = 0; +my $noop = 0; +my $noanno = 0; +my $all = 0; + +my $outdir = '.'; +my $yearly = 0; + +MAIN: { + # Grap command-line options + GetOptions( + "all!" => \$all, + "debug!" => \$debug, + "n!" => \$noop, + "noanno!" => \$noanno, + "outdir=s" => \$outdir, + "verbose!" => \$verbose, + "yearly!" => \$yearly, + + "help" => \$help, + "man" => \$man + ) || usage(0); + usage(1) if $help; + usage(2) if $man; + + $verbose = 1 if $noop; + $verbose = 1 if $debug; + + my @folders = @ARGV; + if (@folders <= 0) { + die qq/Error: No folders specified\n/; + } + + # Trim in leading +'s specified + foreach (@folders) { + s/^\+//; + } + + # Save off current folder + my $curfolder = `folder -fast`; + chomp $curfolder; + print qq/Current folder is "$curfolder"\n"/ if $verbose; + + local(*SCAN); + my $folder; + foreach $folder (@folders) { + print qq/Processing folder "$folder"...\n/ if $verbose; + + # Open read pipe to scan to get message number and date for each + # message. + if (!open(SCAN, qq/scan +$folder -format '/.SCAN_FORMAT_STR.qq/'|/)) { + warn qq/Warning: Non-zero exit status from "scan": $?\n/; + next; + } + + my($msg, $date, $rest); + my($year, $mon, $day); + my($mbox, $packed); + + # Read scan output and copy messages into mailbox files + while () { + next unless /^\s*\d/; + + print $_ if $debug; + chomp; + + ($msg, $date, $rest) = split(' ', $_, 3); + $packed = $date =~ s/\*//; + + if (!$all && $packed) { + print qq/Skipping message "$msg", already packed.\n/; + next; + } + + $date += 1900 if $date < 1900; # just incase + ($year, $mon, $day) = split('-', $date, 3); + + if ($yearly) { + $mbox = $year; + } else { + $mbox = "$year-$mon"; + } + + # Pre-create the mbox file if it does not exist: prevents + # packf from prompting to create. + if (! -e $mbox) { + run_cmd("touch $outdir/$mbox"); + } + + # Have packf actually append message to mailbox file. + if (run_cmd("packf -mbox -file $outdir/$mbox $msg")) { + warn qq/Warning: Non-zero exit status from "packf": $?\n/; + next; + } + + # Annotate message that it has been processed. + if (!$noanno) { + if (run_cmd("anno -component x-mharc-packed -inplace -nodate ". + "-text '1' $msg")) { + warn qq/Warning: Non-zero exit status from "anno": $?\n/; + next; + } + } + } + + close(SCAN); + } + + # Restore current folder + run_cmd("folder +$curfolder"); + +} # End: MAIN + + +sub run_cmd { + if ($verbose) { + print @_, "\n"; + } + return 0 if ($noop); + system(@_); +} + +sub usage { + require Pod::Usage; + my $verbose = shift; + if ($verbose == 0) { + Pod::Usage::pod2usage(-verbose => $verbose); + } else { + my $pager = $ENV{'PAGER'} || 'more'; + local(*PAGER); + my $fh = (-t STDOUT && open(PAGER, "|$pager")) ? \*PAGER : \*STDOUT; + Pod::Usage::pod2usage(-verbose => $verbose, + -output => $fh); + } +} + + +##--------------------------------------------------------------------------## +__END__ + +=head1 NAME + +mh-month-pack - Copy MH/nmh messages into monthly mailbox files. + +=head1 SYNOPSIS + + mh-month-pack [options] folder ... + +=head1 DESCRIPTION + +This program copies MH/nmh messages into mailbox files. By default, +monthly mailbox files are created with filenames of I format. +If the C<-yearly> option is specified, than messages will be split +into yearly, I, files. If a mailbox file already exists, +messages will be appended to it. + +Each MH/nmh process will be annotated inorder to mark the message as +having been processed, unless the C<-noanno> option is specified. +The annotation allows this program to reprocess folders multiple +times and to skip messages that have been packed before. + +This program is provided as part of mharc to provide the ability +to import MH/nmh messages into mharc archives. This program could +be used for sites that have an existing MH/nmh filtering system +inplace and want to usage mharc to generate web-based archives. +This program can be used instead of L to +move incoming messages into the raw message archive before calling +L. + +=head1 OPTIONS + +Any non-option argument is treated as an MH/nmh folder to process. +Folders are specified in the same manner as in MH/nmh commands. + +=over + +=item C<-all> + +Force packing of all messages, even if some messages are marked as +processed. + +=item C<-debug> + +Like C<-verbose>, but prints out more. + +=item C<-help> + +Print out help message. + +=item C<-man> + +Print out the manpage. + +=item C<-n> + +Just echo what would be done, but do not do it. This is handy +to verify what will be done before actually doing it for real. + +=item C<-noanno> + +Do not annotate messages that have been processed. By default, this +program will annotate the messages via the anno(1) command to mark +messages as being processed. Therefore, if the folder is processed +again in the future, the message will be skipped. + +=item C<-outdir> I + +Directory to place mailbox files. If not specified, the +current working directory is used. + +=item C<-verbose> + +Print status of what is going on. + +=item C<-yearly> + +Generate yearly-based mailbox files instead of monthly-based. + +=back + +=head1 DEPENDENCIES + +This program requires that the following MH/nmh commands are in +your search path: C, C, C, C. + +=head1 LIMITATIONS + +=over + +=item * + +Pre-existing gzipped mailbox files are not recognized. + +=back + +=head1 VERSION + +$Id: mh-month-pack,v 1.5 2002/10/11 23:52:28 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/mhonarc-check b/bin/mhonarc-check new file mode 100755 index 0000000..39c1e4d --- /dev/null +++ b/bin/mhonarc-check @@ -0,0 +1,95 @@ +#!/usr/bin/perl +##---------------------------------------------------------------------------## +## File: +## $Id: mhonarc-check,v 1.6 2002/09/15 05:05:54 ehood Exp $ +## Description: +## Checks if mhonarc can be loaded. +##---------------------------------------------------------------------------## +## Copyright (C) 2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##---------------------------------------------------------------------------## + +package MHArc::mhonarc_check; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +eval { + require 'mhamain.pl'; + print "You are using MHonArc v$mhonarc::VERSION\n"; +}; +if ($@) { + print qq/Unable to load MHonArc: $@\n/; + print <<'EOT'; +* If MHonArc is installed, you may need to set the MHONARC_LIB +* variable in lib/config.sh to the directory pathname location +* of MHonArc's library files. +* @INC setting: +EOT + foreach (@INC) { + print qq/\t$_\n/; + } + exit 1; + +} else { + exit 0; +} + +##--------------------------------------------------------------------------## +__END__ + +=head1 NAME + +mhonarc-check - Checks that mharc scripts will be able to load MHonArc + +=head1 SYNOPSIS + + mhonarc-check + +=head1 DESCRIPTION + +This program does a simple check to see that an installation of mharc +can load the MHonArc libraries. If the check fails, the program +returns a non-zero exit status and prints out what can be done to +fix the problem. + +=head1 VERSION + +C<$Id: mhonarc-check,v 1.6 2002/09/15 05:05:54 ehood Exp $> + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mhArc +distribution. + +=cut + diff --git a/bin/mk-procmailrc b/bin/mk-procmailrc new file mode 100755 index 0000000..c95b5da --- /dev/null +++ b/bin/mk-procmailrc @@ -0,0 +1,757 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: mk-procmailrc,v 1.25 2003/08/09 17:51:04 ehood Exp $ +## Description: +## Program to create a procmail recipe file from lists.def. +## Run script with '-man' option to view manpage for this program. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::mk_procmailrc; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::ListDef; +use MHArc::Util qw( usage ); + +my $Verbose = 0; +my $Vfh = \*STDOUT; + +MAIN: { + my @htaccess = (); + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + 'catch-address=s', + 'catch-archive=s', + 'disable-catch-archive!', + 'final-dest=s', + 'home=s', + 'mbox-dir=s', + 'msgid-cache-size=i', + 'out=s', + 'procmail-path=s', + + 'verbose!', + 'help', + 'man' + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + $Verbose = $opt{'verbose'}; + + my $basedir = $opt{'home'} || + $config->{'SW_ROOT'} || + "$Dir/.."; + my $mbox_dir = $opt{'mbox-dir'} || + $config->{'MBOX_DIR'} || + join('/',$basedir,'mbox'); + my $out_file = $opt{'out'} || + $config->{'PROCMAILRC'} || + join('/', $basedir, 'procmailrc.mharc'); + my $procmail_path = $opt{'procmail-path'} || + $config->{'PROCMAIL_PATH'}; + my $cache_size = $opt{'msgid-cache-size'} || + $config->{'MSGID_CACHE_SIZE'} || + 16384; + my $catch_addr = $opt{'catch-address'} || + $config->{'CATCH_ADDRESS'} || + ""; + my $catch_arc = $opt{'catch-archive'} || + $config->{'CATCH_ARCHIVE'} || + '.catch'; + my $nocatch = defined($opt{'disable-catch-archive'}) ? + $opt{'disable-catch-archive'} : + ($config->{'DISABLE_CATCH_ARCHIVE'} || 0); + my $final_dest = $opt{'final-dest'} || + $config->{'FINAL_MSG_DESTINATION'} || + '/dev/null'; + + ## Read lists definition file + my $file = shift(@ARGV) || + $config->{'LISTS_DEF_FILE'} || + "$basedir/lib/lists.def"; + my $listdef = MHArc::ListDef->new($file); + + my $extract_date_prg = join('/', $basedir, 'bin', 'extract-mesg-date'); + + local(*OUTFILE); + my $outfh; + if (!defined($out_file) || ($out_file eq "") || ($out_file eq '-')) { + $outfh = \*STDOUT; + $Vfh = \*STDERR; + } else { + open(OUTFILE, ">$out_file") || + die qq/ERROR: Unable to create "$out_file": $!\n/; + $outfh = \*OUTFILE; + $Vfh = \*STDOUT;; + } + + ## Print procmailrc header + print $outfh <get_names) { + print $Vfh "Generating rule for $name...\n" if $Verbose; + @addr = ( ); + @from_addr = ( ); + + if (defined($listdef->{$name}{'address'})) { + @addr = @{$listdef->{$name}{'address'}}; + } + if (defined($listdef->{$name}{'from-address'})) { + @from_addr = @{$listdef->{$name}{'from-address'}}; + } + + if (!scalar(@addr) && !scalar(@from_addr) && + !defined($listdef->{$name}{'procmail-condition'})) { + # no addresses defined + warn qq/Warning: No addresses or conditions defined for '$name'\n/; + next; + } + + $pm_conditions = ''; + + # create procmail regex for list + if (scalar(@addr) || scalar(@from_addr)) { + $pm_conditions .= '* ('; + if (scalar(@addr)) { + $pm_conditions .= '^TO_'; + $pm_conditions .= '(' if (scalar(@addr) > 1); + $pm_conditions .= join('|', @addr); + $pm_conditions .= ')' if (scalar(@addr) > 1); + $pm_conditions .= '|(^(Delivered-To:|List-Post:).*'; + $pm_conditions .= '(' if (scalar(@addr) > 1); + $pm_conditions .= join('|', @addr); + $pm_conditions .= ')' if (scalar(@addr) > 1); + $pm_conditions .= ')'; + } + if (scalar(@from_addr)) { + $pm_conditions .= '|(' if (scalar(@addr)); + $pm_conditions .= '^From:(.*[^-a-zA-Z0-9_.])?'; + $pm_conditions .= '(' if (scalar(@from_addr) > 1); + $pm_conditions .= join('|', @from_addr); + $pm_conditions .= ')' if (scalar(@from_addr) > 1); + $pm_conditions .= ')' if (scalar(@addr)); + } + $pm_conditions .= ')' . "\n"; + } + + if (defined($listdef->{$name}{'procmail-condition'})) { + foreach $str (@{$listdef->{$name}{'procmail-condition'}}) { + $pm_conditions .= "$str\n"; + } + } + + # check if doing monthly or yearly archives + $period = lc($listdef->{$name}{'period'}[0]) || 'month'; + $period = 'month' if ($name eq $catch_arc); + if ($period eq 'year') { + $time_fmt = '%Y'; + } else { + $time_fmt = '%Y-%m'; + } + + # check if rule should not be final if matched + if ($listdef->{$name}{'final'}) { + $pm_copy = ''; + } else { + $pm_copy = ' c'; + } + + # check if sender specified no archiving should be honored + if ($listdef->{$name}{'check-no-archive'}) { + $noarchive =<<'EOT'; + + :0 + * ^(X-no-archive: yes|Restrict: no-external-archive) + /dev/null +EOT + } else { + $noarchive = ""; + } + + # check if separating out cvs commits + if (($check_cvs = $listdef->{$name}{'cvs-commits'}[0]) && + ($pm_conditions =~ /\S/)) { + my $cvs_prefix = $listdef->{$name}{'cvs-subject-prefix'}[0] || + 'CVS commit'; + print $outfh < program. The procmailrc is generated +from Cmharc-rootE/lib/lists.def>. + +This program is typically invoked by calling C from +the mharc root directory with configuration options specified +in Cmharc-rootE/lib/config.sh>. + +=head1 LIST DEFINITION FILE + +The list definition file, Cmharc-rootE/lib/lists.def>, +controls how the procmailrc used by L +is generated. The format of the file is simple and more convenient +than writing the procmailrc file yourself. + +The basic format of the file is as follows: + +=over + +=item * + +Any blank links or lines starting with a C<#> are ignored. + +=item * + +Lines with the following format: + + Option-Name: Option-Value + +is an option. + +=back + +=head2 lists.def Supported Options + +=over + +=item Name + +Starts, and defines, the name of an archive. This name serves +as the directory name containing archive data and the list title +(the C<$LIST-TITLE$> MHonArc resource variable) for archive pages. +A common practice is to use the list address, but this is not required, +especially if the archive is a combination of multiple lists, or it +is prefered to use a more abstract name for simplicity. + +The name also is used when the L script looks for an +archive specific MHonArc resource settings. If the following +resource file exists, + + $MHA_RC_DIR/.mrc + +where C<$MHA_RC_DIR> is the value of the MHA_RC_DIR C variable +and CnameE> is the name of the archive, L will +pass the resource file to MHonArc when processing the HTML archive. +This provides a convenient way to provide list-specific customization +to the archive. + +=item Address + +Mail address of the list. +Multiple C
options can be specified +for an archive if a list has more than one known address (e.g. due +to migration) or the archive is a collection of multiple lists. + +B The address is technically treated as part of +a procmail regular expression. Take the following as an example: + + Address: mharc-users@mhonarc.org + +In regular expressions, the '.' character represents any character. +Therefore, the following strings would match the above specification: + + mharc-users@mhonarc#org + mharc-users@mhonarcXorg + mharc-users@mhonarc@org + +In practice, this technicality will generally have no affect, but if +you want to be pendantic, do the following: + + Address: mharc-users@mhonarc\.org + +The '\' tells procmail to treat the '.' literally. + +Because the actual address string given is treated as part of a regular +expression, you can specify a range of addresses with a single option. +For example, + + mharc-[^@]*@mhonarc\.org + +will match the following addresses: + + mharc-users@mhonarc.org + mharc-rules@mhonarc.org + mharc-rocks@mhonarc.org + mharc-is-the-best@mhonarc.org + ... + +=item All-Lists-Name + +Label to use for Name column of all-lists index. If not specified, +the name provided by the C option is used. + +=item Check-No-Archive + +Boolean option (C<0> or C<1>) if author specified archiving permission +is honored. The author can request no archiving should be done +by defining one of the the following header fields: + + Restrict: no-external-archive + X-no-archive: yes + +If C is enabled, a message to the list with either +field defined will not be archived. + +=item CVS-Commits + +Boolean option (C<0> or C<1>) if CVS commit messages should be +archived separately. + +Use this option for development lists that have CVS project commits +mailed to the list, and you want to avoid cluttering regular discussion +mail. + +=item CVS-Subject-Prefix + +Specifies the C prefix denoting CVS commits to the list. +This option is only used if C is specified. + +=item Description + +Brief description of archive and serves as the main title of archive +index pages. + +=item Final + +Boolean option (C<0> or C<1>) if generated rule should be final. +I.e. If a message matches, further rules will not be examined. + +Use this option to short-circuit messages from being stored in multiple +archives. For example, you may want to catch messages cross-posted +to a special address to only be archived in a special archive and +not any of the regular archives. + +Another example is if you use the special C "C<.catch>" +(or the C<-catch-archive> setting described in L<"OPTIONS">). +Using "C<.catch>" is handy for C definitions to pre-catch +messages that should not be placed in any list archive. + +=item From-Address + +Mail address of the list. This option is an alternative to C
+where a list can only be donoted by the C field of messages. +This is fairly common for one-way lists, like newsletters, where +subscribers receive list messages but cannot post to the list. + +Multiple C options can be specified if a list has more +than one known address (e.g. due to migration) or the archive is a +collection of multiple lists. + +=item Hide-From-All-Lists + +Boolean option (C<0> or C<1>) if archive should not be listed in +the all-lists index. +If set to C<1>, the archive will not be shown. +The default value for this option is C<0>. + +=item Lang + +Sets the language/locale of the archive. + +B Only set this option if using a version of MHonArc +(v2.6.7, or later) that supports the LANG resource. Otherwise, +you will get a runtime error when L is +invoked. + +See the LANG resource reference page of the MHonArc documentation +for more information. + +=item MHonArc-Options + +Additional MHonArc command-line options. + +=item No-Raw-Link + +Boolean option (C<0> or C<1>) if links to raw archives should exist. +If set to C<1>, links will not be created or disabled. The +default value for this option is C<0>. + +Use this option if your HTML archives have been customized to obscure +addresses to prevent address harvesting. + +=item No-Search + +Boolean option (C<0> or C<1>) if searching should be disabled. +If set to C<1>, no search index is created and the C<$SEARCH-FORM$> +custom MHonArc resource variable will be set to the empty string. + +B Disable searching will diable some navigational features +that are dependent upon the search index. + +=item Period + +If archive is a yearly or monthly archive. Legal values are C +or C. If Period is not defined, C is the default. + +=item Procmail-Condition + +Additional condition to apply to base address check. The condition +must be legal procmailrc syntax and should include any prefixing C<*>, +C, et. al. This option can be specified multiple times. + +C can also be used inplace of C
and +C to provide arbitrary matching rules for archiving +messages. This is useful for rare cases where messages to be archived +cannot be determined by receipient or from addresses. + +B Exercise caution when using this option, especially if +C is true. When C is true, an additionaly +rule already exists to check for the C setting. + +=back + +=head2 lists.def Notes + +=over + +=item * + +Every archive defined must define at least one C
, C, +or C option. + +=item * + +The order of archive definitions is mirrored in the generated procmail +rcfile. This is important to properly honor the sematics of archives +with the C option specified. + +=back + +=head2 lists.def Example + + ## In this definition, we define multiple addresses to check. + Name: mhonarc-users + Description: MHonArc Users + Address: mhonarc-users@mhonarc.org + Address: mhonarc@ncsa.uiuc.edu + Address: mhonarc@rosat.mpe-garching.mpg.de + + ## This definition defines a list that receives CVS commits and those + ## commits should be separated into a special archive as to not + ## pollute the discussion messages with cvs commit messages + Name: mhonarc-dev + Description: MHonArc Development + Address: mhonarc-dev@mhonarc.org + CVS-Commits: 1 + CVS-Subject-Prefix: CVS: + +=head1 OPTIONS + +You should never have to invoke this program with any options since +C specifies any options used by this program. However, +for advanced uses, or you do not care if you mess things up, the +following options are available: + +=over + +=item C<-catch-address> I + +The name of the email address to forward all unmatched message to. +This is an alternative to C<-catch-archive>, and will supercede +C<-catch-archive>, if defined. +If this option is not specified, the C variable in +C is used. + +=item C<-catch-archive> I + +The name of the I archive. The I archive collects +all messages that do not match any list rules. If this +option is not specified, the C variable in +C is used, else the name "C<.catch>" is used. + +B If you use this option, it is recommended that the name +starts with a C<.> (a dot). This insures that no search index is built +and it will not be listed in the all-lists page. + +=item C<-disable-catch-archive> + +If specified, no I archive will be defined. + +B Care should be used when using this option since any +message that does not match a normal rule will be lost. + +=item C<-final-dest> I + +The destination of messages that make it to the end of the procmailrc. +It is normal for messages to make it to the end since the list matching +rules create copies of the message during filtering. Hence, it is +normal to see "C" destinations in the procmail log +and it does not indicate that a message was lost. + +Message copying is done inorder to properly archive a message that has +been cross-posted to multiple lists. Message copying is not done for +archives with the C option set to 1, for CVS commit archives, +or for messages that are captured by the catch archive. + +This option is generally only used for debugging purposes. + +If C<-final-dest> is not specified, the +C variable in C is used, else +C is used. + +=item C<-help> + +Print out usage information. + +=item C<-home> I + +B. + +Root pathname of software and archives. If not specified, +C variable in C is used, else the parent directory +that containing this program is used. + +=item C<-man> + +Print out entire manpage. + +=item C<-mbox-dir> I + +Root pathname containing raw mailbox archives. If not specified, +C variable in C is used, else C/mbox> +is used. + +=item C<-msgid-cache-size> I + +The maximum size, in bytes, of the message-id cache. The message-id +cache helps avoid processing duplicate messages. + +If this option is not specified, the C variable +in C is used, else 16384 will be used. + +=item C<-out> I + +Output filename. If this option is not specified, the C +variable in C is used, else C/procmailrc.mharc> +is used. + +If "-" is the I, then the procmailrc will be dumped to +standard out. + +=item C<-procmail-path> I + +The search path for C to use. If this option is not +specified, the C variable in C is used. + +=back + +=head1 FILES + +=over + +=item Cmharc-rootE/lib/lists.def> + +Mailing lists definition file. + +=item Cmharc-rootE/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 VERSION + +$Id: mk-procmailrc,v 1.25 2003/08/09 17:51:04 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/mk-search-index b/bin/mk-search-index new file mode 100755 index 0000000..df2a3f7 --- /dev/null +++ b/bin/mk-search-index @@ -0,0 +1,65 @@ +#!/bin/sh +##--------------------------------------------------------------------------## +## File: +## $Id: mk-search-index,v 1.4 2002/09/13 07:25:55 ehood Exp $ +## Description: +## Create/update search index for list. The name of the +## list is specified on the command-line. +## +## NOTE: This script is probably no longer needed since +## web-archive calls the search indexing tool directly. +##--------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +if [ ! -z "$GATEWAY_INTERFACE" ]; then + echo "CGI use FORBIDDEN!" + exit 1 +fi + +if [ $# -ne 1 ]; then + echo "Invalid invocation." + exit 1 +fi +list=$1 + +PATH=$HOME/bin:/usr/local/bin:/usr/bin:/bin; export PATH +BASEDIR=`dirname $0` + +# Load configuration +. $BASEDIR/../lib/config.sh + +# Sanity checks +if [ ! -e "$MKNMZ_RC" ]; then + echo "$MKNMZ_RC does not exist!" + exit 1 +fi +if [ ! -d "$MKNMZ_TMPL_DIR" ]; then + echo "$MKNMZ_TMPL_DIR is not a directory!" + exit 1 +fi +if [ ! -d "$HTML_DIR/$list" ]; then + echo "$HTML_DIR/$list is not a directory!" + exit 1 +fi + +$MKNMZ --mhonarc \ + -f "$MKNMZ_RC" \ + -T "$MKNMZ_TMPL_DIR" \ + -O "$HTML_DIR/$list" \ + "$HTML_DIR/$list" diff --git a/bin/read-mail b/bin/read-mail new file mode 100755 index 0000000..3a5d02c --- /dev/null +++ b/bin/read-mail @@ -0,0 +1,166 @@ +#!/usr/bin/perl +##--------------------------------------------------------------------------## +## File: +## $Id: read-mail,v 1.10 2002/09/15 03:33:08 ehood Exp $ +## Description: +## Read mail spool and archive messages. +##--------------------------------------------------------------------------## +## Copyright (C) 2001 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##--------------------------------------------------------------------------## + +package MHArc::read_mail; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use MHArc::Util qw( ch_dir cmd usage ); + +MAIN: { + # Grap command-line options + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + 'force!', + 'home=s', + 'verbose!', + + 'help', + 'man', + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + my $verbose = $opt{'verbose'}; + if ($verbose) { + $MHArc::Util::ECHO_CMDS = 1; + } + + my $home = $opt{'home'} || + $config->{'SW_ROOT'} || + "$Dir/.."; + my $force = $opt{'force'}; + + ch_dir($home) || + die qq/ERROR: Unable to change directory to "$home": $!\n/; + + if (-e '.noarchive') { + print "Archiving is currently disabled\n" if $verbose; + if (!$force) { + exit 0; + } + print "However, -force specified, so continuing...\n" if $verbose; + } + + my @cmd_args = (); + push(@cmd_args, '-verbose') if $verbose; + if (cmd('./bin/filter-spool', @cmd_args) == 0) { + cmd('./bin/web-archive', @cmd_args); + } +} + +##---------------------------------------------------------------------------## +__END__ + +=head1 NAME + +read-mail - Archive incoming mail + +=head1 SYNOPSIS + + read-mail + read-mail [options] + +=head1 DESCRIPTION + +This program is part of mharc and is the main program for archiving +mail. It is generally called via cron. + +This program does very little itself, but it invokes other mharc +scripts to do all the work. + +If the file C<.noarchive> exists in mharc root directory, then this +program will do nothing and exit (unless the C<-force> option is +specified). This is to allow one to disable incoming mail processing +while performing administrative tasks. + +=head1 OPTIONS + +This program is generally called without any command-line options +since it will read Cmharc-rootE/lib/config.sh> for any configurable +options. Regardless, the following command-line options are +available: + +=over + +=item C<-help> + +Print out usage information. + +=item C<-home> I + +Root pathname of archiving software and data. If not specified, +C variable in C is used, else the parent directory +that contains this program is used. + +=item C<-man> + +Print out entire manpage. + +=item C<-verbose> + +Print out status messages. + +=back + +=head1 FILES + +=over + +=item C/lib/config.sh> + +Main configuration file for mharc. + +=back + +=head1 VERSION + +$Id: read-mail,v 1.10 2002/09/15 03:33:08 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/bin/web-archive b/bin/web-archive new file mode 100755 index 0000000..5043266 --- /dev/null +++ b/bin/web-archive @@ -0,0 +1,1153 @@ +#!/usr/bin/perl +##---------------------------------------------------------------------------## +## File: +## $Id: web-archive,v 1.44 2003/08/09 17:56:05 ehood Exp $ +## Description: +## Updates/creates web archives from mailbox archives. +## Run script with '-man' option to view manpage for this program. +##---------------------------------------------------------------------------## +## Copyright (C) 2001-2002 Earl Hood +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +## 02111-1307, USA +##---------------------------------------------------------------------------## + +package MHArc::web_archive; + +##--------------------------------------------------------------------------## +# +BEGIN { die qq/CGI use FORBIDDEN!\n/ if (defined($ENV{'GATEWAY_INTERFACE'})); } +my $Dir; BEGIN { $Dir = `dirname $0`; chomp $Dir; } +use lib "$Dir/../lib"; # Add relative lib to search path +# +##--------------------------------------------------------------------------## +# +use MHArc::Config; +my $config = MHArc::Config->load("$Dir/../lib/config.sh"); +# +##--------------------------------------------------------------------------## + + +use Getopt::Long; +use POSIX; +use MHArc::ListDef; +use MHArc::Util qw( usage ); + +# Load MHonArc library +require 'mhamain.pl'; + +my $debug = 0; + +# Regular expression to match mail folder/mboxes +my $folder_regex = '\d+(?:-\d+)?'; + +MAIN: { + my %opt = ( ); + my $clstatus = GetOptions(\%opt, + 'alllistsurl=s', # Root to all lists URL. + 'alllistsfile=s', # Pathname to all lists index page. + 'debug|verbose', # Show what is going on in detail. + 'editidx', # Edit archive pages; useful to apply MHonArc resource + # changes. + 'editallidx', # Regen all lists index. + 'editidxonly', # Edit archive index pages only. + 'editrootidx', # Regen top index. + 'home=s', # Pathname of home directory of archive account. + 'htmldir=s', # Root directory for html archives. + 'htmlurl=s', # Root URL for html archives. + 'keepsearch!', # Keep search index on a rebuild. + 'listsdef=s', # Pathname to list definition file. + 'mboxdir=s', # Root directory for mbox archives. + 'mboxurl=s', # Root URL for mbox archives. + 'mharc=s', # MHonArc resource file for archives. + 'mhamaxsize=i', # Maximum MHonArc archive size. + 'mhapagesize=i', # Maximum MHonArc index page size. + 'mknmz=s', # Pathname to Namazu make search index program. + 'mknmzrc=s', # Pathname to Namazu configuration file. + 'mknmztmpldir=s', # Pathname to Namazu template directory. + 'mesgcgi=s', # Message CGI URL. + 'mnavcgi=s', # Month navigation CGI URL. + 'mtimeage=i', # Modify time age of a mailbox file to be considered + # for processing. + 'nosearch', # Do not update search indexes. + 'rebuild', # Rebuild archives from scratch. + 'rooturl=s', # Root URL to archives. + 'searchcgi=s', # Search CGI URL. + + 'man', + 'help' + ); + usage(0) unless $clstatus; + usage(1) if $opt{'help'}; + usage(2) if $opt{'man'}; + + my $HOME = $opt{'home'} || + $config->{'SW_ROOT'} || + "$Dir/.."; + my $ROOT_URL = $opt{'rooturl'} || + $config->{'ROOT_URL'} || + "/~mhonarc/archives"; + my $LISTS_DEF_FILE = $opt{'listsdef'} || + $config->{'LISTS_DEF_FILE'} || + "$HOME/lib/lists.def"; + my $HTML_DIR = $opt{'htmldir'} || + $config->{'HTML_DIR'} || + "$HOME/html"; + my $HTML_URL = $opt{'htmlurl'} || + $config->{'HTML_URL'} || + "$ROOT_URL/html"; + my $MBOX_DIR = $opt{'mboxdir'} || + $config->{'MBOX_DIR'} || + "$HOME/mbox"; + my $MBOX_URL = $opt{'mboxurl'} || + $config->{'MBOX_URL'} || + "$ROOT_URL/mbox"; + my $INFO_DIR = $opt{'infodir'} || + $config->{'INFO_DIR'} || + "$HOME/info"; + my $INFO_URL = $opt{'infourl'} || + $config->{'INFO_URL'} || + "$ROOT_URL/info"; + my $MHA_RC = $opt{'mharc'} || + $config->{'MHA_RC'} || + "$HOME/lib/common.mrc"; + my $MHA_RC_DIR = $opt{'mharcdir'} || + $config->{'MHA_RC_DIR'} || + "$HOME/lib/mrc"; + my $MHA_MAXSIZE = $opt{'mhamaxsize'} || + $ENV{'WA_MAXSIZE'} || + 2000; + my $MHA_PAGESIZE = $opt{'mhapagesize'} || + $ENV{'WA_PAGESIZE'} || + 200; + my $MTIME_AGE = $opt{'mtimeage'} || + $ENV{'WA_MTIME_AGE'} || + $config->{'MTIME_AGE'} || + 86400; + my $MKNMZ = $opt{'mknmz'} || + $config->{'MKNMZ'} || + '/usr/local/bin/mknmz'; + my $MKNMZRC = $opt{'mknmzrc'} || + $config->{'MKNMZ_RC'} || + "$HOME/cgi-bin/mknmzrc"; + my $MKNMZTMPLDIR = $opt{'mknmztmpldir'} || + $config->{'MKNMZ_TMPL_DIR'} || + "$HOME/cgi-bin/template", + my $ALL_LISTS_URL = $opt{'alllistsurl'} || + $config->{'ALL_LISTS_URL'} || + $HTML_URL; + my $MESG_CGI = $opt{'mesgcgi'} || + $config->{'MESG_CGI'} || + join('/', $ROOT_URL,'cgi-bin/mesg.cgi'); + my $MNAV_CGI = $opt{'mnavcgi'} || + $config->{'MNAV_CGI'} || + join('/', $ROOT_URL,'cgi-bin/mnav.cgi'); + my $SEARCH_CGI = $opt{'searchcgi'} || + $config->{'SEARCH_CGI'} || + join('/', $ROOT_URL,'cgi-bin/namazu.cgi'); + my $EXTRACT_CGI = $opt{'extractchcgi'} || + $config->{'EXTRACT_CGI'} || + join('/', $ROOT_URL,'cgi-bin/extract-mesg.cgi'); + + my $rebuild = $opt{'rebuild'} || + $ENV{'WA_REBUILD'} || 0; + my $keepsearch = $opt{'keepsearch'}; + my $editidx = $opt{'editidx'} || + $ENV{'WA_EDIT'} || 0; + my $editidxonly = $opt{'editidxonly'} || 0; + my $editrootidx = $opt{'editrootidx'}; + my $editallidx = $opt{'editallidx'}; + my $nosearch = $opt{'nosearch'} || + $ENV{'WA_NOSEARCH'} || 0; + $debug = $opt{'debug'} || + $ENV{'WA_DEBUG'}; + + my $all_index = $opt{'alllistsfile'} || + $config->{'ALL_LISTS_FILE'} || + join('/', $HTML_DIR, 'lists.html'); + my $main_header = $config->{'MAIN_HEADER'} || + join('/', $HTML_DIR, '.PNM.head'); + my $main_footer = $config->{'MAIN_FOOTER'} || + join('/', $HTML_DIR, '.PNM.foot'); + + my $time = time; + if ($rebuild) { + $editidx = 0; + $editrootidx = 0; + $editallidx = 0; + } else { + $keepsearch = 0; + } + $editidx = 1 if $editidxonly; + if ($editidx) { + $editrootidx = 0; + $editallidx = 0; + } + + my $listdef = MHArc::ListDef->new($LISTS_DEF_FILE); + print "Loaded lists definitions.\n" if $debug; + + if ($editallidx) { + # Just updating all-lists index + update_archive_index( + '-config' => $config, + '-listdef' => $listdef, + '-htmldir' => $HTML_DIR, + '-htmlurl' => $HTML_URL, + '-infodir' => $INFO_DIR, + '-infourl' => $INFO_URL, + '-allindex' => $all_index + ); + last MAIN; + } + + mhonarc::initialize(); + print "MHonArc initialized.\n" if $debug; + + local(*DIR, *INDEX, *FILE); + + print "Reading $MBOX_DIR.\n" if $debug; + opendir(DIR, $MBOX_DIR) || die qq/Unable to open "$MBOX_DIR": $!/; + my @dirs = (); + + # Get list of archives to process + if (@ARGV) { + # list of archives specified on the command-line + @dirs = @ARGV; + } else { + # read mbox dir to get list + @dirs = grep { (-d "$MBOX_DIR/$_") && + ($_ ne '.') && + ($_ ne '..') + } readdir(DIR); + closedir(DIR); + } + + my(@months, @folders); + my($dir, $list, $mon, $mondir, $htmldir, $cvs, $title, $mtime, + $folder, $i, $yr, $prevdir, $nextdir, $prevmon, $nextmon, + $disable_search, $listname, $short_title); + + print "Lists: ", join(', ', @dirs), "\n" if $debug; + foreach $list (@dirs) { + print "Processing $list ...\n" if $debug; + + @folders = (); + $listname = $list; + $cvs = ($listname =~ s/\.CVS$//); + + if (!$editidx && !$editrootidx) { + # Get list of input mailboxes to process + + $dir = join('/', $MBOX_DIR, $list); + if (!opendir(DIR, $dir)) { + warn qq/Unable to open "$dir": $!/; + next; + } + + # create .noraw file indicator if no-raw-link specified + my $no_raw_file = join('/', $dir, '.noraw'); + my $no_raw_htaccess = join('/', $dir, '.htaccess'); + if ($listdef->{$listname}{'no-raw-link'}[0]) { + if (! -e $no_raw_file) { + local(*NORAW); + if (!open(NORAW, ">$no_raw_file")) { + warn qq/Warning: Unable to create "$no_raw_file": $!\n/; + } else { + close(NORAW); + } + } + if (! -e $no_raw_htaccess) { + local(*HTACCESS); + if (!open(HTACCESS, ">$no_raw_htaccess")) { + warn qq/Warning: Unable to create "$no_raw_htaccess": $!\n/; + } else { + print HTACCESS 'Order allow,deny', "\n", + 'Deny from all', "\n"; + close(HTACCESS); + } + } + } elsif (-e $no_raw_file) { + if (!unlink($no_raw_file)) { + warn qq/Warning: Unable to remove "$no_raw_file": $!\n/; + } + } + + @months = grep { /^$folder_regex(?:\.gz)?$/o } readdir(DIR); + closedir(DIR); + print "Mboxes: ", join(', ', @months), "\n" if $debug; + + foreach $mon (@months) { + $mondir = join('/', $dir, $mon); + if ($rebuild) { + push(@folders, $mondir); + next; + } + $mtime = (stat($mondir))[9]; + print "$mondir mtime: $mtime\n" if $debug; + if (($time - $mtime) < $MTIME_AGE) { + push(@folders, $mondir); + } + } + + next if (!@folders); + print "Folders: ", join(', ', @folders), "\n" if $debug; + + } elsif ($editidx) { + # Just editing pages so we get folder list from html directory + $dir = join('/', $HTML_DIR, $list); + if (!opendir(DIR, $dir)) { + warn qq/Unable to open "$dir": $!/; + next; + } + @months = grep { /^$folder_regex$/o } readdir(DIR); + closedir(DIR); + + foreach $mon (@months) { + $mondir = join('/', $dir, $mon); + push(@folders, $mondir); + } + next if (!@folders); + print "Editidx Folders: ", join(', ', @folders), "\n" if $debug; + } + @folders = reverse sort @folders; + + $htmldir = join('/', $HTML_DIR, $list); + if ($rebuild) { + clean_html_archive($htmldir, $keepsearch); + } + mkdir($htmldir, 0777); + + $disable_search = ($list =~ /^\./) || + ((defined($listdef->{$listname}{'no-search'}) && + $listdef->{$listname}{'no-search'}[0])); + + if (defined($listdef->{$listname}{'description'})) { + $title = join(' ', @{$listdef->{$listname}{'description'}}); + } else { + $title = $listname; + } + $short_title = $listname; + if ($cvs) { + $title = '[CVS] '.$title; + $short_title = '[CVS] '.$short_title; + } + + if (!$editrootidx) { + # define arguments to mhonarc + my @mhaargs = ( + '-modtime', + '-lockmethod', 'flock', + #'-maxsize', $MHA_MAXSIZE, + #'-idxsize', $MHA_PAGESIZE, + '-rcfile', $MHA_RC, + #'-outdir' , $htmldir, + '-title', "$title (date)", + '-ttitle', "$title (thread)", + '-definevar', "LIST-TITLE='$short_title'", + '-definevar', "LIST-NAME='$list'", + '-definevar', "SEARCH-CGI=$SEARCH_CGI", + '-definevar', "PNAV-CGI=$MNAV_CGI", + '-definevar', "EXTRACT-CGI=$EXTRACT_CGI", + '-definevar', "MESG-CGI=$MESG_CGI", + '-definevar', "ALL-LISTS-URL=$ALL_LISTS_URL", + + '-definevar', "MNAV-CGI=$MNAV_CGI", # backwards compatibility + ); + + if (defined($listdef->{$listname}{'lang'})) { + push(@mhaargs, '-lang', $listdef->{$listname}{'lang'}[0]); + } + + if (-e "$MHA_RC_DIR/$list.mrc") { + push(@mhaargs, '-rcfile', "$MHA_RC_DIR/$list.mrc"); + } + if ($cvs) { + push(@mhaargs, '-nothread'); + push(@mhaargs, '-definevar', "THREAD-IDX-LINK=''"); + } else { + push(@mhaargs, '-thread'); + } + if ($list =~ /^\./) { + push(@mhaargs, + '-nothread', + '-definevar', "SEARCH-FORM=''"); + push(@mhaargs, '-definevar', "THREAD-IDX-LINK=''"); + } + if ($editidx) { + push(@mhaargs, '-editidx'); + push(@mhaargs, '-nomsgpgs') if $editidxonly; + } + if (defined($config->{'MSG_DATE_FIELDS'})) { + push(@mhaargs, '-datefields', $config->{'DATE_FIELDS'}); + } + if (!$debug && !$rebuild) { + push(@mhaargs, '-quiet'); + } + if (!$rebuild && !$editidx) { + push(@mhaargs, '-add'); + } + if ($listdef->{$listname}{'check-no-archive'}) { + push(@mhaargs, '-checknoarchive'); + } + + # add any custom options specified in definition file + if (defined($listdef->{$listname}{'mhonarc-options'})) { + require 'shellwords.pl'; + push(@mhaargs, + shellwords(join(' ', @{$listdef->{$listname}{'mhonarc-options'}}))); + } + + # if searching is disabled, zero-out $SEARCH-FORM$ + if ($disable_search) { + push(@mhaargs, '-definevar', "SEARCH-FORM=''"); + } + + my(@fmhaargs); + @months = ( ); + foreach $folder (@folders) { + ($mon = $folder) =~ s/\.gz$//; + $mon =~ s/^.*\///; + push(@months, $mon); + } + + my $cur_msg_cnt; + for ($i=0; $i < @folders; ++$i) { + $folder = $folders[$i]; + $mon = $months[$i]; + $mondir = join('/', $htmldir, $mon); + + # make sure directory exists + mkdir($mondir, 0777); + + # set final arguments to mhonarc + @fmhaargs = ( + @mhaargs, + '-outdir', $mondir, + '-definevar', "CUR-PERIOD='$mon'", + + '-definevar', "CUR-MONTH='$mon'", # backwards compatibility + ); + push(@fmhaargs, $folder) unless $editidx; + + # call mhonarc + print "Processing archive $mondir...\n" if $debug; + print "\tmhonarc options: ", join(' ', @fmhaargs), "\n" if $debug; + if (!mhonarc::open_archive(@fmhaargs)) { + warn qq/Warning: Unable to open "$mondir" archive: /, + qq/($mhonarc::CODE) $mhonarc::ERROR\n/; + next; + } + $mhonarc::CBRcVarExpand = \&mha_rcvar_expand; + $cur_msg_cnt = $mhonarc::NumOfMsgs || 0; + mhonarc::process_input(); + if ($mhonarc::CODE != 0) { + warn qq/Warning: Problem processing "$mondir": /, + qq/($mhonarc::CODE) $mhonarc::ERROR\n/; + next; + } + if ($cur_msg_cnt == $mhonarc::NumOfMsgs) { + print "Skipping search index, no new messages in archive\n" + if $debug; + next; + } + + # update search index + # The -Y option is used so we do not have to process all months + # to update index. + if (!$keepsearch && !$nosearch && !$disable_search) { + my @nmzargs = ( + $MKNMZ, + '--mhonarc', # only do mhonarc pages + '-f', $MKNMZRC, # specify resource file + '-T', $MKNMZTMPLDIR, # specify template directory + '-O', $htmldir, # specify location to place index + '-Y' # do not delete existing files + ); + if (!$debug && !$rebuild) { + push(@nmzargs, '--quiet'); + } + push(@nmzargs, $mondir); + print "Search Index Command: ", join(" ", @nmzargs), "\n" if $debug; + + if (system(@nmzargs)) { + warn qq/Warning: Non-zero exit status returned from /, + qq/"@nmzargs": $?\n/; + } + namazu_cleanup($htmldir); + } + } + } + + ## Update monthly index + if (!opendir(DIR, $htmldir)) { + warn qq/Warning: Unable to open $htmldir for reading: $!\n/; + next; + } + @months = reverse sort grep { /^$folder_regex/o } readdir(DIR); + print "Month listing for main index: @months\n" if $debug; + closedir(DIR); + my $indexhtml = join('/', $htmldir, 'index.html'); + if (!open(INDEX, ">$indexhtml.tmp")) { + warn qq/Warning: Unable to open $htmldir for reading: $!\n/; + next; + } + + my @vars = ( + '-nosearch' => $disable_search, + 'SEARCH-CGI' => $SEARCH_CGI, + 'LIST-TITLE' => $short_title, + 'LIST-NAME' => $list, + 'LIST-DESC' => $title, + ); + print_template(\*INDEX, $main_header, @vars); + print INDEX "
    \n"; + foreach $mon (@months) { + print INDEX qq|
  • $mon:|; + print INDEX qq|  [Date]| + if (-e join('/', $htmldir, $mon, 'index.html')); + print INDEX qq|  [Thread]| + if (-e join('/', $htmldir, $mon, 'threads.html')); + + if (!$listdef->{$listname}{'no-raw-link'}[0]) { + my $raw_label = '[Raw: ]'; + my $compressed = 0; + my $mbox_file = join('/', $MBOX_DIR, $list, $mon); + my $mbox_url = join('/', $MBOX_URL, $list, $mon); + if (! -e $mbox_file) { + $mbox_file .= ".gz"; + $mbox_url .= ".gz"; + $compressed = 1; + } +# if (-e $mbox_file) { +# print INDEX qq|  [mbox: |, +# (-s _), qq| bytes|; +# print INDEX qq|, gzipped| if $compressed; +# print INDEX qq|]|; +# } + } + print INDEX qq|
    • \n|; + } + print INDEX "
\n"; + print_template(\*INDEX, $main_footer, @vars); + close(INDEX); + if (!rename("$indexhtml.tmp", $indexhtml)) { + warn qq|Warning: Unable to rename "$indexhtml.tmp" to |, + qq|"$indexhtml": $!\n|; + } + } + + update_archive_index( + '-config' => $config, + '-listdef' => $listdef, + '-htmldir' => $HTML_DIR, + '-htmlurl' => $HTML_URL, + '-infodir' => $INFO_DIR, + '-infourl' => $INFO_URL, + '-allindex' => $all_index + ); + +} # End: MAIN + +############################################################################ + +sub entify { + my $str = shift; + $str =~ s/\&/\&/; + $str =~ s//\>/; + $str; +} + +sub get_periods { + my $dir = shift; + + local(*DIR); + if (!opendir(DIR, $dir)) { + warn qq/Warning: Unable to open "$dir": $!/; + return ( ); + } + my @months = reverse sort grep { /^$folder_regex$/o } readdir(DIR); + closedir(DIR); + @months; +} + +sub read_template { + my $fh = shift; + my %varhash = @_; + my $data = ""; + + if ($varhash{'-nosearch'}) { + local $_; + my $ignore = 0; + while (<$fh>) { + if ($ignore) { + $ignore = 0 if //; + next; + } + if (//) { + $ignore = 1; + next; + } + s/\$([^\$]+)\$/$varhash{$1}/ge; + $data .= $_; + } + } else { + local $/; + $data = <$fh>; + $data =~ s/\$([^\$]+)\$/$varhash{$1}/ge; + } + $data; +} + +sub print_template { + my $fhout = shift; + my $file = shift; + if (-e $file) { + print "Reading template file $file\n" if $debug; + local(*FILE); + if (open(FILE, $file)) { + print $fhout read_template(\*FILE, @_); + close(FILE); + } else { + warn qq/Warning: Unable to open "$file": $!\n/; + } + } +} + +sub namazu_cleanup { + my $dir = shift; + my $lock = join('/', $dir, 'NMZ.lock2'); + local(*LOCK); + if (!open(LOCK, $lock)) { + # no lock file left around, so everything should be okay + return; + } + my $pid = ; + close(LOCK); + if (!kill(0, $pid)) { + warn qq/Warning: Stale "$lock", removing it\n/; + if (!unlink($lock)) { + warn qq/Warning: Unable to remove "$lock": $!\n/; + } + } +} + +sub format_date { + my $time = shift; + my $fmt = shift || '%Y-%m-%d %H:%M:%S'; + #my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($time); + #$year += 1900; ++$mon; + #sprintf("%d-%02d-%02d %02d:%02d:%02d", $year,$mon,$mday,$hour,$min,$sec); + POSIX::strftime($fmt, localtime($time)); +} + +## Retrieve the last time an archive was updated. +# We scan the message pages since their mtime should be set to the +# date of the message. +# +sub retrieve_last_update { + my $archive = shift; + + local(*DIR); + if (!opendir(DIR, $archive)) { + warn qq/Warning: Unable to open "$archive" for reading: $!\n/; + return undef; + } + + my $latest = 0; + my $mtime = 0; + my $file; + foreach $file (readdir(DIR)) { + next unless $file =~ /^msg\d+\.html/; + $mtime = (stat(join('/', $archive, $file)))[9]; + $latest = $mtime if ($mtime > $latest); + } + close(DIR); + + if ($latest == 0) { + # No luck with message pages, so try database file + warn qq/Warning: Unable to determine last update from message pages /, + qq/for "$archive"\n/; + if (-e join('/', $archive, '.mhonarc.db')) { + $latest = ((stat(_))[9]); + } elsif (-e join('/', $archive, 'mhonarc.db')) { + $latest = ((stat(_))[9]); + } + } + if ($latest == 0) { + # No luck with data, so use directory mtime + $latest = ((stat($archive))[9]); + } + + $latest; +} + +## Remove HTML archive +# +sub clean_html_archive { + my $dir = shift; # Directory of archive + my $ks = shift; # Flag is search index files should be preserved + if (!$ks) { + # delete everything + print "Removing $htmldir\n" if $debug; + system('/bin/rm', '-rf', $dir); + return; + } + + # keep search index, so must delete each period sub-directory + local(*DIR); + opendir(DIR, $dir) || + die qq/ERROR: Unable to open "$dir" for reading: $!\n/; + my @subdirs = map { join('/',$dir,$_) } + grep { /^$folder_regex$/o } readdir(DIR); + closedir(DIR); + my $subdir; + foreach $subdir (@subdirs) { + print "Removing $subdir\n" if $debug; + system('/bin/rm', '-rf', $subdir); + } +} + +## Retrieve the list info URL. +# +sub get_info_url { + my %opts = @_; + my $pathname = join('/', $opts{'-dir'}, $opts{'-name'}) . '.html'; + if (! -e $pathname) { + return undef; + } + join('/', $opts{'-baseurl'}, $opts{'-name'}) . '.html'; +} + +## Generats the all-lists index. +# +sub update_archive_index { + print "Generating root archive index...\n" if $debug; + + my %opts = @_; + my $config = $opts{'-config'}; + my $listdef = $opts{'-listdef'}; + my $html_dir = $opts{'-htmldir'}; + my $html_url = $opts{'-htmlurl'}; + my $info_dir = $opts{'-infodir'}; + my $info_url = $opts{'-infourl'}; + my $index_html = $opts{'-allindex'}; + + my $header = $opts{'-header'} || + $config->{'ALL_LISTS_HEADER'} || + join('/', $html_dir, '.PNM.all-head'); + my $footer = $opts{'-footer'} || + $config->{'ALL_LISTS_FOOTER'} || + join('/', $html_dir, '.PNM.all-foot'); + + my $label_name = $opts{'-label-name'} || + $config->{'ALL_LISTS_LABEL_NAME'} || + 'Name'; + my $label_indexes + = $opts{'-label-indexes'} || + $config->{'ALL_LISTS_LABEL_INDEXES'} || + 'Current Index'; + my $label_last = $opts{'-label-last-updated'} || + $config->{'ALL_LISTS_LABEL_LAST_UPDATED'} || + 'Last Updated'; + my $label_info = $opts{'-label-info'} || + $config->{'ALL_LISTS_LABEL_INFO'} || + '[info]'; + my $label_date = $opts{'-label-date'} || + $config->{'ALL_LISTS_LABEL_DATE'} || + '[Date]'; + my $label_threads + = $opts{'-label-threads'} || + $config->{'ALL_LISTS_LABEL_THREADS'} || + '[Threads]'; + my $time_fmt = $opts{'-time-fmt'} || + $config->{'ALL_LISTS_DATE_FORMAT'} || + '%Y-%m-%d %H:%M:%S'; + + local(*IDX); + my $tmp_index = $index_html . ".tmp"; + if (!open(IDX, ">$tmp_index")) { + warn qq/Warning: Unable to create "$tmp_index": $!\n/; + return; + } + + my %updated = ( ); + my($list, $listname, $last_updated, $dir, $latest, $info); + + foreach $listname (keys %$listdef) { + next if $listname =~ /^\./; # skip hidden archives + next if $listdef->{'hide-from-all-lists'}[0]; + + foreach $list ($listname, "$listname.CVS") { + $dir = join('/', $html_dir, $list); + next unless -e $dir; + + print "Computing last update for $list...\n" if $debug; + my @months = get_periods($dir); + next unless @months; + $latest = $months[0]; + $last_updated = retrieve_last_update(join('/', $dir, $latest)); + if (!defined($last_updated)) { + print "Unable to compute last update for $list.\n" if $debug; + next; + } + $updated{$list} = [ $last_updated, $list, $listname, $dir, $latest ]; + } + } + + print_template(\*IDX, $header); + print IDX qq|\n|, + qq|\n|, + qq||, + qq||, + qq||, + qq|\n\n|; + + my($time); + foreach $list (sort { $updated{$b}->[0] <=> $updated{$a}->[0] } + keys(%updated)) { + ($time, $list, $listname, $dir, $latest) = @{$updated{$list}}; + + print "Printing listing for $list\n" if $debug; + $last_updated = format_date($time, $time_fmt); + $last_updated =~ s/ /\ /g; + my $short_title = entify($listdef->{$listname}{'all-lists-name'}[0] || + $listname); + my $description = entify($listdef->{$listname}{'description'}[0] || + $listname); + if ($list =~ /\.CVS$/) { + $short_title .= " (CVS)"; + $description .= " (CVS commits)"; + } + $info = get_info_url( + '-name' => $listname, + '-dir' => $info_dir, + '-baseurl' => $info_url + ); + + print IDX qq|\n|; + + print IDX qq|\n|; + + + print IDX qq|\n|; + + print IDX qq|\n|; + } + + print IDX qq|
$label_name$label_indexes$label_last
 |, + qq|$short_title |; + print IDX qq|$label_info | + if defined($info); + print IDX qq||; + print IDX qq| $label_date | + if (-e join('/', $dir, $latest, 'index.html')); + print IDX qq| $label_threads | + if (-e join('/', $dir, $latest, 'threads.html')); + print IDX qq| |, $last_updated, qq| 
\n|; + print_template(\*IDX, $footer); + close(IDX); + + if (!rename($tmp_index, $index_html)) { + warn qq/Warning: Unable to rename "$tmp_index" to "$index_html": $!\n/; + } +} + +sub mha_rcvar_expand { + my $mha_index = shift; + my $var_name = shift; + my $arg = shift; + + my $val = undef; + if ($var_name eq 'NMZ-SUBJECT-QUERY') { + my($lref, $key, $pos) = + mhonarc::compute_msg_pos($mha_index, $var_name, $arg); + return undef unless defined($key); + + my $clipped = 0; + $val = mhonarc::get_base_subject($key); + if (length($val) > 128) { + $val = substr($val, 0, 128); + $clipped = 1; + } + $val = "\Q$val\E"; + $val =~ s/(?:\\\s)+/\\s+/g; # \Q will escape whitespace + my $repl_re = $mhonarc::SubReplyRxp; + my $query = "+subject:/^(?:$repl_re)*$val"; + $query .= '\s*$' unless $clipped; + $query .= '/'; + return ($query, 0, 0); + } + + return undef; +} + + +############################################################################ +__END__ + +=head1 NAME + +web-archive - Update/create MHonArc archives from mailbox archives. + +=head1 SYNOPSIS + + web-archive + web-archive [options] + web-archive [options] [list-name ...] + +=head1 DESCRIPTION + +This program is part of mharc and has the responsibility of processing +the mailbox archives created by the L script to +update and/or create MHonArc archives. + +This program is automatically called by the L script for +processing incoming mail within the mail spool if L +returns with an okay status. However, this program can be manually +invoked to rebuild archives, edit existing archives, or other +administrative tasks. Since there may be a need to do selective archive +processing, any non-option related argument is treated as mailing +list archive name to process. + +=head1 OPTIONS + +=over + +=item C<-alllistidx> I + +Pathname of file to generate the all lists index. +If not specified, the value of the C variable in +C is used, else it defaults to "C/lists.html>". + +=item C<-alllistsurl> I + +URL to page containing list of all mailing lists archived. +If not specified, the value of the C variable in +C is used, else it defaults to C<-htmlurl>. + +=item C<-editidx> + +Edit archive pages, useful to apply MHonArc resource +changes. + +=item C<-editrootidx> + +Only regenerate root index pages for archives. This is useful if +you make changes to the C<.PNM.head> or C<.PNM.foot> files that you +want immediately applied. + +=item C<-help> + +Print out usage information. + +=item C<-home> I + +Root pathname of archiving software and data. +If not specified, the parent directory that contains this program +is used. + +=item C<-htmldir> I + +Root directory for html archives. +If not specified, "C/html>" is used. + +=item C<-htmlurl> I + +URL root to HTML archives. +If not specified, defaults to C/html>. + +=item C<-infodir> I + +Pathname of directory containing informational pages for each list +archive. Information for a list archive can be provided by creating a +file called "C.html>". Once created, a link to the file +(based on the value of the C<-infourl> option) will be generated in +the all-lists index to it. + +If this option is not specified, the value of the C variable +in C is used, else it defaults to "C/info>". + +=item C<-infourl> I + +Base URL containing informational pages for each list archive. +If not specified, the value of the C variable in +C is used, else it defaults to "C/info>". + +=item C<-keepsearch> + +Preserve search index if C<-rebuild> is specified. This option +is handy if all that is desired is to rebuild the HTML archives +from the raw data since the overhead of rebuilding the search indexes +will be avoided. + +B Do not use C<-keepsearch> if you have removed messages +from the raw mail archives since resulting HTML message pages may +have different URIs than what is stored within the search index. + +=item C<-listsdef> I + +Pathname to mailing lists definition file. +If not specified, "C/lib/lists.def>" is used. + +=item C<-man> + +Print out entire manpage. + +=item C<-mboxdir> I + +Root directory for mbox archives. +If not specified, "C/mbox>" is used. + +=item C<-mharc> I + +MHonArc resource file for archives. +If not specified, "C/lib/common.mrc>" is used. + +=item C<-mharcdir> I + +Directory containing list-specifc MHonArc resource files. A given +list archive can have additional resource settings by creating a +file called C.mrc> within the directory specified by +C<-mharcdir>. + +If C<-mharcdir> is not specified, "C/lib/mrc>" is used. + +=item C<-mhamaxsize> I + +Maximum MHonArc archive size. +If not specified the value of the C environment variable is used. + +=item C<-mhapagesize> I + +Maximum MHonArc index page size. +If not specified the value of the C environment variable is used. + +=item C<-mknmz> I + +Pathname to Namazu make search index program. +If not specified, "C" is used. + +=item C<-mknmzrc> I + +Pathname to Namazu configuration file. +If not specified, "C/cgi-bin/mknmzrc>" is used. + +=item C<-mknmztmpldir> I + +Pathname to Namazu template directory. +If not specified, "C/cgi-bin/template>" is used. + +=item C<-mnavcgi> I + +URL to monthly navigation cgi program. +If not specified, C/cgi-bin/mnav.cgi> is used. + +=item C<-mtimeage> I + +Modify time age of a mailbox file to be considered for processing. If +not specified the value of the C environment variable is +used. + +=item C<-nosearch> + +Do not update search indexes. + +=item C<-rebuild> + +Rebuild archives from scratch. + +=item C<-rooturl> I + +URL root of archives. +If not specified, C is used. + +=item C<-searchcgi> I + +URL to search cgi program. +If not specified, C/cgi-bin/namazu.cgi> is used. + +=item C<-verbose> + +Show what is going on in detail. + +=back + +=head1 ENVIRONMENT + +Environment variable usage is deprecated. + +The following environment variables are recognized: + +=over + +=item C + +If set to a true value, detailed information of progress will be +printed to stdout. Debugging can also be enabled by the +C<-debug> command-line option. + +=item C + +If set to a true value, archives will be editted. It is probably +better to use the C<-editidx> command-line option instead if archives +editing is desired. + +=item C + +Maximum MHonArc archive size. The default value is 2000. This setting +can be overridden by the C<-mhamaxsize> command-line option. + +=item C + +The modification age, in seconds, for a mailbox to be considered +for processing. The default value is C<86400> (one day). +This setting can be overridden by the C<-mtimeage> command-line option. + +=item C + +If set to a true value, the Namazu search indexes will not be updated +for archives processed. Disabling of search index updates can also be +disabled by the C<-nosearch> command-line option. + +=item C + +MHonArc index page size. The default value is 200. This setting +can be overridden by the C<-mhapagesize> command-line option. + +=item C + +If set to a true value, archives will be rebuilt. It is probably +better to use the C<-rebuild> command-line option instead if rebuilding +is desired. + +=back + +=head1 VERSION + +$Id: web-archive,v 1.44 2003/08/09 17:56:05 ehood Exp $ + +=head1 AUTHOR + +Earl Hood, earl@earlhood.com + +This program is part of the mharc archiving system and comes with +ABSOLUTELY NO WARRANTY and may be copied only under the terms of +the GNU General Public License, which may be found in the mharc +distribution. + +=cut + diff --git a/html b/html new file mode 120000 index 0000000..eaef0bd --- /dev/null +++ b/html @@ -0,0 +1 @@ +/arc/mharc-html \ No newline at end of file diff --git a/mbox b/mbox new file mode 120000 index 0000000..d1d1b29 --- /dev/null +++ b/mbox @@ -0,0 +1 @@ +/arc/mharc-mbox \ No newline at end of file diff --git a/procmailrc.mailman b/procmailrc.mailman new file mode 100644 index 0000000..765bbd4 --- /dev/null +++ b/procmailrc.mailman @@ -0,0 +1,47 @@ +# Mostly autocreated +# Hand tuned for mailman lists - $LISTNAME is the name of the list to archive +# +##====================================================================== +##!!!!!!!!!!!!!!!!!!!!!! AUTO-CREATED, DO NOT EDIT !!!!!!!!!!!!!!!!!!!!! +##====================================================================== +## Procmail resource file for mharc archives generated from +## /home/mharc/lib/lists.def +## by ./bin/mk-procmailrc +##====================================================================== + +SHELL=/bin/sh +LINEBUF=4096 +UMASK=133 +PATH=/home/mharc/bin:/usr/local/bin:/bin:/usr/bin +BASEDIR=/home/mharc + +# the log has grown to 400MB and is completely useless -- bernie +#LOGFILE=/dev/null +LOGFILE=$BASEDIR/log/procmail.log + +## Do alot of logging? +#VERBOSE=yes + +## Should deliveries be logged? +LOGABSTRACT=yes + +## Root path to mail folders +MBOXROOT=/home/mharc/mbox + +##====================================================================== +## Start Rules +##====================================================================== + +## Avoid duplicate messages +#:0 Wh: $BASEDIR/msgid.lock +#| formail -D 16384 $BASEDIR/msgid.cache + +## mailman lists +:0 Wi +CMDEXC_=| mkdir -m 755 -p $MBOXROOT/$LISTNAME + +:0 Wi +MESGDATE_=| /home/mharc/bin/extract-mesg-date -fmt '%Y-%m' + +:0: +$MBOXROOT/$LISTNAME/$MESGDATE_ -- 2.25.1