Added the user documentation.

README.md

@@ -0,0 +1,44 @@
+# ASL Rulebook 2
+
+<img align="right" src="doc/features/images/asl-rulebook2.small.png">
+This program lets you search through the ASL Rulebook index, and jump directly to the rules you're looking for.
+
+Click [here](doc/features/) for more details.
+<br clear="all">
+
+With [some work](doc/extend.md), you can also:
+- add rules for other modules
+- have Q+A and errata automatically pop-up when you click on a rule
+- include your own annotations
+- include the ASOP, complete with clickable links to each rule
+
+*NOTE: This project integrates with my other [`asl-articles`](https://github.com/pacman-ghost/asl-articles) project, so that if an article references a rule, it becomes a clickable link that will open a browser showing that rule.*
+
+### Installation
+
+After cloning the repo, you can either:
+- run it using Docker (recommended, run `./run-container.sh`)
+- run it from source (requires Python 3.8.7, install the module, run `asl_rulebook2/webapp/run_server.py`)
+
+*NOTE: Run either command with no arguments to get help.*
+
+Run either command with a `--port` argument, then connect to the server in a browser e.g.
+
+``` ./run-container.sh --port 5020 ```
+
+Then open `http://localhost:5020`.
+
+*NOTE: The program requires Firefox; Chrome doesn't really support a key feature it needs.*
+
+A few things need to be set up before the program can be used; the webapp will guide you through the process.
+
+*NOTE: If you are running from source, you will also need Ghostscript installed.*
+
+### Preparing the data files
+
+The first time the program is run, the MMP eASLRB PDF must be analyzed, and some key data extracted and prepared. The webapp will do this for you automatically, but in the event there are problems, [this page](doc/prepare.md) describes how to do it manually.
+
+### FAQ
+
+- Why is this project called ASL Rulebook *2*? <br> *Several years ago, I wrote a similar *ASL Rulebook* project that worked from a scanned copy of the ASLRB. Since it required a prepared version of the PDF, which couldn't be distributed, there was no point releasing the source code. When MMP released their official eASLRB, I updated the code to work with that, and have released it here.*
+- Why doesn't the sidebar update (e.g. to show Q+A) when I click on links within the PDF itself? <br> *This is due to the way the program is architected. The PDF is shown in an iframe, and so the outer application can't get event notifications for things that happen inside that iframe. I might revisit this later (but it's a *lot* of work :-/).*

asl_rulebook2/webapp/config/constants.py

@@ -2,7 +2,7 @@
 
 import os
 
-APP_NAME = "ASL Rulebook"
+APP_NAME = "ASL Rulebook 2"
 APP_VERSION = "v0.1" # nb: also update setup.py
 APP_DESCRIPTION = "Search engine for the ASL Rulebook."
 

asl_rulebook2/webapp/static/prepare.js

@@ -136,6 +136,9 @@ gPrepareApp.component( "upload-panel", {
             <div class="info"> You <u>must</u> use the <a href="https://www.wargamevault.com/product/344879/Electronic-Advanced-Squad-Leader-Rulebook" target="_blank">offical MMP eASLRB</a>. <br>
                 A scanned copy of a printed RB <u>will not work</u>!
             </div>
+            <div class="info"> If there are problems here, you can try to prepare <br>
+                your data files <a href="https://github.com/pacman-ghost/asl-rulebook2/blob/master/doc/prepare.md" target="_blank">manually</a>.
+            </div>
         </div>
     </div>
 </div>`,

asl_rulebook2/webapp/tests/fixtures/full/README.md

@@ -0,0 +1,57 @@
+This directory contains a set of files that demonstrate the full functionality of the program. There is only a minimal amount of content, but there's a little bit of everything, and enough to give you a feel for what the program can do. In particular, there are only a few targets defined, so most of the ruleid's will be greyed out (since the program doesn't know about them).
+
+There are no PDF's provided, so you won't see any real content in the content pane. However, if you have already prepared the MMP eASLRB PDF, copy it into this directory (with the name `ASL Rulebook.pdf`), and everything will automagically work. Alternatively, add `?no-content=1` to the URL when opening the webapp in your browser.
+
+Once you've got the hang of how things work, you can use the files here as a basis to start building the real data files.
+
+### Getting started
+
+The search engine covers:
+- any index files (e.g. the `.index` files)
+- any Q+A (look in the `q+a/` directory)
+- any errata (look in the `errata/` directory)
+- any user annotations (defined in the `annotations.json` file)
+- most of the ASOP (look in the `asop/` directory)
+
+Take a look through those data files to get a feel for what you can search for.
+
+Run the program, specifying this directory as the data directory e.g.
+```
+./run-container.sh --port 5020 \
+    --data asl_rulebook2/webapp/tests/fixtures/full/ \
+    --asop asl_rulebook2/webapp/tests/fixtures/asop/asop/
+```
+
+*NOTE: You will see a few things in the UI that might look wrong; this is because this data set is used for testing the program, including error conditions and bad data.*
+
+### Q+A
+
+Search for *"encirclement"*; note how the content PDF has jumped directly to rule A7.7, and the Q+A for that rule have automatically appeared. Click on the image to zoom in. Click on the "close" icon, or press Escape to dismiss the Q+A.
+
+You will see the search results in the left panel, which can be filtered using the checkboxes at the top of the panel.
+
+If you click on the O6.7 rule link, the content pane will switch to showing the *Red Barricades* PDF (if it were there), providing seamless switching between multiple PDF files.
+
+### Errata
+
+Search for *"CCPh"*, and an errata that has been written for rule A3.8 will pop up. Note that it doesn't appear in the search results, because it doesn't contain the word *"CCPh"*. However, if you search for *"errata"*, you will get both the A3.8 index entry (CCPh), and the errata, because they both contain the word *"errata"*.
+
+### User annotations
+
+You can also add your own annotations for a specific rule, defined in the `annotations.json` file.
+
+Search for *"WP"* for an example.
+
+### ASOP
+
+Search for *"CC"* - the last search result will be the 8.21B ASOP step, *"DURING LOCATION's CCPh"*. Click on the titlebar to go to the ASOP entry itself.
+
+You can also just browse through the ASOP, by clicking on the ASOP icon in the bottom-left corner.
+
+### Footnotes
+
+Search for *"error"*. The A.2 rule will come up, and because it has an associated footnote, this will appear in a popup. Hover your mouse over the balloon if you need time to read it.
+
+### Chapters
+
+Finally, if you just want to browse through the rulebook(s), you can quickly jump to major sections withing each chapter by clicking on the *Chapters* icon in the bottom-left corner. Note how *Red Barricades* and *Kampfgruppe Scherer* have been merged in with the main rulebook, to provide a single view of all the configured rulebooks.

doc/extend.md

@@ -0,0 +1,41 @@
+# Adding more data
+
+### Adding an MMP module
+
+To add rules for a module already referenced by the ASLRB index (e.g. RB or BR:T), you first need to provide a scanned PDF of the rules, named `ASL Rulebook (xxx).pdf`.
+
+Then, write a targets file `ASL Rulebook (xxx).targets`, that describes where each rule lives within the PDF. As an example, take a look at the `ASL Rulebook.targets` file that was extracted for you.
+
+*NOTE: Only the page number is required, the X/Y position on the page is optional.*
+
+Finally, bookmarks need to be created in the PDF for each rule, so that the program can jump directly to each rule:
+```
+    asl_rulebook2/bin/prepare_pdf.py \
+        xxx-original.pdf \
+        --targets targets.json \
+        --yoffset 5 \
+        --output /tmp/xxx-prepared.pdf
+```
+Save the prepared file as `ASL Rulebook (xxx).pdf`, and the targets file, in your data directory, and restart the server.
+
+Optionally, you can also provide:
+- a `ASL Rulebook (xxx).chapters` file (to be able to browse the PDF in the *Chapters* panel)
+- a `ASL Rulebook (xxx).footnotes` file (if the rules have any footnotes).
+
+To add a chapter icon and background, create files `XXX-icon.png` and `XXX-background.png`, where `XXX` is the chapter ID. For MMP modules, this is inferred from the rule ID (e.g. "O3.3" becomes "O"); for third-party modules, you can define a chapter ID by adding a `chapter_id` key to the `.chapters` files.
+
+### Adding a third-party module
+
+To add rules for a module not already referenced by the ASLRB index, the process is the same as above, but you also need to write a `.index` file. As an example, take a look at the `ASL Rulebook.index` file that was extracted for you.
+
+All the files should have the same base filename e.g.
+- kgs.index
+- kgs.pdf
+- kgs.targets
+- etc...
+
+### Adding Q+A, errata, user annoations, ASOP
+
+This is described [here](../asl_rulebook2/webapp/tests/fixtures/full/).
+
+*NOTE: If you add Q+A, there is a tool in `$/asl_rulebook2/bin/qa-helper/` to help with the process.*

doc/features/global.css

@@ -0,0 +1,8 @@
+.ruleid { font-family: monospace ; }
+
+.info {
+    margin-top: 10px ; min-height: 25px ;
+    padding-left: 30px ; background: no-repeat url(info.png) ;
+    font-size: 90% ; font-style: italic ; color: #444 ;
+}
+

doc/features/imageZoom/jquery.imageZoom.css

@@ -0,0 +1,48 @@
+div.jquery-image-zoom {
+	line-height: 0;
+	font-size: 0;
+
+	z-index: 10;
+
+	border: 5px solid #fff;
+    background: #eee; /* TM 25jan15: Added this to make it easier to see images with transparent backgrounds. */
+	margin: -5px;
+
+	-webkit-box-shadow: 0 0 10px rgba(0, 0, 0, 0.3);
+	-moz-box-shadow: 0 0 10px rgba(0, 0, 0, 0.3);
+	box-shadow: 0 0 10px rgba(0, 0, 0, 0.3);
+}
+
+	div.jquery-image-zoom a {
+		background: url(jquery.imageZoom.png) no-repeat;
+
+		display: block;
+		width: 25px;
+		height: 25px;
+
+		position: absolute;
+		left: -17px;
+		top: -17px;
+		/* IE-users are prolly used to close-link in right-hand corner */
+		*left: auto;
+		*right: -17px;
+
+		text-decoration: none;
+		text-indent: -100000px;
+		outline: 0;
+
+		z-index: 11;
+	}
+
+	div.jquery-image-zoom a:hover {
+		background-position: left -25px;
+	}
+
+	div.jquery-image-zoom img, 
+	div.jquery-image-zoom embed, 
+	div.jquery-image-zoom object, 
+	div.jquery-image-zoom div {
+		width: 100%;
+		height: 100%;
+		margin: 0;
+	}

doc/features/imageZoom/jquery.imageZoom.min.js

@@ -0,0 +1 @@
+jQuery.fn.imageZoom=function(c,b){var a=c.extend({speed:200,dontFadeIn:1,hideClicked:1,imageMargin:30,className:"jquery-image-zoom",loading:"Loading..."},b);a.doubleSpeed=a.speed/4;c(document).keydown(function(d){if(d.keyCode==27){c("div.jquery-image-zoom a").click()}});return this.click(function(k){var h=c(k.target);var g=h.is("a")?h:h.parents("a");g=(g&&g.is("a")&&g.attr("href").search(/(.*)\.(jpg|jpeg|gif|png|bmp|tif|tiff)$/gi)!=-1)?g:false;var i=(g&&g.find("img").length)?g.find("img"):false;c("div.jquery-image-zoom a").click();if(g){g.oldText=g.text();g.setLoadingImg=function(){if(i){i.css({opacity:"0.5"})}else{g.text(a.loading)}};g.setNotLoadingImg=function(){if(i){i.css({opacity:"1"})}else{g.text(g.oldText)}};var d=g.attr("href");if(c("div."+a.className+' img[src="'+d+'"]').length){return false}var j=function(l){g.setNotLoadingImg();var u=i?i:g;var q=i?a.hideClicked:0;var p=u.offset();var n={width:u.outerWidth(),height:u.outerHeight(),left:p.left,top:p.top};var o=c('<div><img src="'+d+'" alt=""/></div>').css("position","absolute").appendTo(document.body);var m={width:l.width,height:l.height};var s={width:c(window).width(),height:c(window).height()};if(m.width>(s.width-a.imageMargin*2)){var r=s.width-a.imageMargin*2;m.height=(r/m.width)*m.height;m.width=r}if(m.height>(s.height-a.imageMargin*2)){var t=s.height-a.imageMargin*2;m.width=(t/m.height)*m.width;m.height=t}m.left=(s.width-m.width)/2+c(window).scrollLeft();m.top=(s.height-m.height)/2+c(window).scrollTop();var e=c('<a href="#">Close</a>').appendTo(o).hide();if(q){g.css("visibility","hidden")}o.addClass(a.className).css(n).animate(m,a.speed,function(){e.fadeIn(a.doubleSpeed)});var v=function(){e.fadeOut(a.doubleSpeed,function(){o.animate(n,a.speed,function(){g.css("visibility","visible");o.remove()})});return false};o.click(v);e.click(v)};var f=new Image();f.src=d;if(f.complete){j(f)}else{g.setLoadingImg();f.onload=function(){j(f)}}return false}})};

doc/features/images/.gitignore

@@ -0,0 +1 @@
+/snags/

doc/features/index.html

@@ -0,0 +1,77 @@
+<!DOCTYPE html>
+<!-- IMPORTANT! This needs to be on the gh-pages branch to be viewable as HTML! :-/
+        FIXME! Or we might be able to do in the repo's Settings.
+-->
+<html lang="en">
+
+<head>
+<meta charset="utf-8">
+<title> ASL Rulebook 2 features </title>
+<link rel="stylesheet" type="text/css" href="imageZoom/jquery.imageZoom.css" />
+<link rel="stylesheet" type="text/css" href="global.css" />
+</head>
+
+<body>
+
+<p> <img src="images/search-heat.png" class="imageZoom" style="float:right;width:40%;margin-left:1em;">
+<h2> Basic features </h2>
+<p> Out-of-the-box, this program gives you full-text search over the ASLRB index.
+<p> It will jump to the exact position in the eASLRB PDF for a rule when you click on a search result.
+<p> And if the rule has any associated footnotes, these will be shown in a popup.
+
+<p style="margin-top:1.5em;"> You can also browse the ASLRB by chapter: <br>
+<img src="images/chapters.png" class="imageZoom" style="margin:1em 0 0 2em;width:300px;">
+
+<br><br>
+
+
+<p> <img src="images/search-cellar.png" class="imageZoom" style="float:right;width:40%;margin-left:1em;">
+<h2> Adding rules for other modules </h2>
+<p> Once you've got the program up and running, you can then think about extending it. It's a lot of work, but the results are insanely cool!
+<p> To the right, I searched for "cellar", and the program has found results from <em>Red Barricades</em>. The rules for this are referenced in the ASLRB index, but the content is not yet in the MMP eASLRB.
+<p> However, I added a PDF scan of the rules, plus information about where each rule is within that PDF (a <em>"targets file"</em>), and so when I click on the <span class="ruleid">O3.3</span> search result, it seamlessly opens the <em>Red Barricades</em> PDF and jumps to that rule.
+<br clear="all"> <br>
+
+<p> <img src="images/search-air-support.png" class="imageZoom" style="float:right;width:40%;margin-left:1em;">
+You can also include third-party modules that are not referenced in the ASLRB index, in this case, LFT's <em>Kampfgruppe Scherer</em>. I added an index for it, which is then also searched. As with <em>Red Barricades</em>, I also added a PDF scan of the rules, and a targets file that specifies the location of each rule, and clicking on the <span class="ruleid">KGS CG9</span> search result takes me directly to that rule in the KGS PDF.
+<br>
+
+<p> I also added information about chapters in the PDF's, so that I can browse through them in the usual way: <br>
+<img src="images/chapters-extended.png" class="imageZoom" style="margin:1em 0 0 2em;width:30%;">
+
+<br><br>
+
+
+<p> <img src="images/ruleinfo-encircled.png" class="imageZoom" style="float:right;width:30%;margin-left:1em;">
+<h2> Showing Q+A and errata </h2>
+Q+A and errata can also be included. This is a <em>lot</em> of work, but the results are amazing. If you click on a rule that has Q+A and/or errata associated with it, they will be shown, alongside the rule you're looking for.
+<p> Here, I've searched for <em>"encircled"</em>, and the program has automatically shown Q+A and errata for rule <span class="ruleid">A7.7</span>.
+<div class="info"> Note that this errata is actually obsolete, since it's already been incorporated into the current MMP eASLRB, but it's shown here as an example. </div>
+<br clear="all">
+
+<p> <img src="images/search-encircled.png" class="imageZoom" style="float:right;width:30%;margin-left:1em;">
+Going back to the search results, you can see these Q+A entries included in there.
+<p> Note the checkboxes underneath the search box, that let you filter the results by type.
+<br clear="all">
+
+
+<p> <img src="images/user-anno.png" class="imageZoom" style="float:left;width:30%;margin-right:1em;">
+<h2> User annotations </h2>
+You can also add your own notes to the search engine.
+<p> Here, I've added a note about what <em>Majority Squad Type</em> means, and a link back to the Game Squad post that talks about it.
+<br clear="all">
+
+
+<p> <img src="images/asop.png" class="imageZoom" style="float:right;width:40%;margin-left:1em;">
+<h2> Advanced Sequence Of Play </h2>
+Finally, the ASOP can also be included, with clickable links to each of the referenced rules.
+<p> ASOP entries are also included in search results: <br>
+<img src="images/asop-sr.png" class="imageZoom" style="margin:1em 0 0 2em;width:30%;">
+
+</body>
+
+<script src="jquery-3.6.0.min.js"></script>
+<script src="imageZoom/jquery.imageZoom.min.js"></script>
+<script src="utils.js"></script>
+
+</html>

doc/features/utils.js

@@ -0,0 +1,20 @@
+$(document).ready( () => {
+    // make images zoomable
+    makeImagesZoomable( $("body") ) ;
+} ) ;
+
+// --------------------------------------------------------------------
+
+function makeImagesZoomable( $elem )
+{
+    // look for images that have been marked as zoomable, and make it so
+    $elem.find( "img.imageZoom" ).each( function() {
+        $(this).wrap( $( "<a>", {
+            class: "imageZoom",
+            href: $(this).attr( "src" ),
+            title: "Click to zoom",
+            onFocus: "javascript:this.blur()"
+        } ) ) ;
+    } ) ;
+    $elem.find( "img.imageZoom" ).imageZoom( $ ) ;
+}

doc/prepare.md

@@ -0,0 +1,59 @@
+# Preparing the data files
+
+The first time the program is run, the MMP eASLRB PDF must be analyzed (to extract some key data), and prepared. The webapp will do this for you automatically, but since there are different versions of the eASLRB (and everyone gets a different file anyway, because of the watermarks), it's possible that the automatic process might fail.
+
+In this case, you can try to prepare the data files manually, which gives you a chance to fix any problems.
+
+The steps below assume that you've created a directory `/tmp/prepared/` to store the prepared files, and `$EASLRB` refers to your copy of the MMP eASLRB PDF file.
+
+*NOTE: If you are running the program using Docker, you will need to perform these steps inside the container.*
+
+### Extract from the eASLRB PDF
+
+The first step is to extract the information we need from the eASLRB PDF.
+```
+    asl_rulebook2/extract/all.py $EASLRB \
+        --format json \
+        --save-index /tmp/prepared/ASL\ Rulebook.index \
+        --save-targets /tmp/prepared/ASL\ Rulebook.targets \
+        --save-chapters /tmp/prepared/ASL\ Rulebook.chapters \
+        --save-footnotes /tmp/prepared/ASL\ Rulebook.footnotes \
+        --progress
+```
+This extracts the information we need, and saves it in the 4 data files.
+
+### Prepare the PDF
+
+Next, we need to prepare the eASLRB PDF, namely create bookmarks for each rule, so that the webapp can jump directly to each one:
+```
+    asl_rulebook2/bin/prepare_pdf.py \
+        $EASLRB \
+        --targets /tmp/prepared/ASL\ Rulebook.targets \
+        --yoffset 5 \
+        --output /tmp/prepared.pdf \
+        --compression ebook \
+        --progress
+```
+We also take the opportunity to compress the PDF.
+
+### Fixup the PDF
+
+Finally, we need to fixup some issues in the PDF:
+```
+    asl_rulebook2/bin/fixup_mmp_pdf.py \
+        /tmp/prepared.pdf \
+        --rotate \
+        --optimize-web \
+        --output /tmp/prepared/ASL\ Rulebook.pdf \
+        --progress
+```
+This fixes the zoom problem when clicking on links within the PDF, and rotates any landscape pages.
+
+### Using the prepared files
+
+You should now have 5 files (the 4 extracted data files, plus the fixed-up PDF), which can be passed in to the program e.g.
+```
+    ./run-container.sh \
+        --port 5020 \
+        --data /tmp/prepared/
+```