# Reading Library

Welcome to **chanmainvest/reading_library**, the investment reading pipeline for Chanma Investment. This repository contains mirrors, catalogs, and curated text versions of core books, research papers, and educational resources related to energy, commodities, financial history, investing, trading, banking, and markets.

This repository is optimized primarily for **AI agent ingestion** — book content is stored as markdown (`books/<slug>/index.md`) — but it also provides a premium, highly responsive, human-readable offline browsing experience via GitHub Pages.

👉 **[Access the Live Portal on GitHub Pages](https://chanmainvest.github.io/reading_library/)**

### 🌐 Catalog Portal Features
The redesigned catalog portal includes:
- **Light/Dark Mode Toggle**: Persistent theme switcher bubble (Sun/Moon icons) styling both the catalog and the SPA reader/on-device chatbot.
- **Standardized 3D Book Cards**: A visual grid of dynamic, realistic virtual book covers with creases, page edges, custom gradients, and emblems.
- **Advanced Search & Filters**: Search catalog contents, filter dynamically by tag categories (e.g., Energy, Investing, Economics), sort by Title/Date, or toggle "Available to Read Only".
- **Dynamic JSON Architecture**: Rendered entirely from a single source of truth database [books.json](books.json) with a safe offline-browse fallback.

---

## 📚 Currently Mirrored Materials

1. **Oil 101** (`/books/oil101`)
   - An offline mirror of Morgan Downey's *Oil 101*, the definitive guide to the oil industry.
   - Access: [Browse Oil 101](./index.html#/books/oil101) or via [GitHub Pages Portal](https://chanmainvest.github.io/reading_library/#/books/oil101).
   
2. **NatGas 101** (`/books/natgas101`)
   - An offline mirror of Morgan Downey's *NatGas 101*, the complete guide to North American natural gas markets, infrastructure, and geology.
   - Features a programmatically compiled, offline-friendly responsive SVG chart of the **Duck Curve** in Chapter 12 ("Power Generation").
   - Access: [Browse NatGas 101](./index.html#/books/natgas101) or via [GitHub Pages Portal](https://chanmainvest.github.io/reading_library/#/books/natgas101).

## 📖 Finance & Markets Catalog

Requested finance and markets books are tracked in [books/catalog.json](./books/catalog.json). EPUB files with explicit rights are converted into the same published markdown format as the web mirrors.

Converted from local ebook source (rights-approved), published as `books/<slug>/index.md`:

1. **Antifragile** — *Nassim Nicholas Taleb* (`/antifragile`)
   - Access: [Browse Antifragile](./index.html#/books/antifragile)
2. **Beating the Street** — *Peter Lynch* (`/beating-the-street`)
   - Access: [Browse Beating the Street](./index.html#/books/beating-the-street)
3. **Central Banking 101** — *Joseph Wang* (`/central-banking-101`)
   - Access: [Browse Central Banking 101](./index.html#/books/central-banking-101)
4. **Common Stocks and Uncommon Profits** — *Philip A. Fisher* (`/common-stocks-and-uncommon-profits`)
   - Access: [Browse Common Stocks and Uncommon Profits](./index.html#/books/common-stocks-and-uncommon-profits)
5. **Debt** — *David Graeber* (`/debt`)
   - Access: [Browse Debt](./index.html#/books/debt)
6. **Flash Boys** — *Lewis, Michael* (`/flash-boys`)
   - Access: [Browse Flash Boys](./index.html#/books/flash-boys)
7. **Going Infinite** — *Michael Lewis* (`/going-infinite`)
   - Access: [Browse Going Infinite](./index.html#/books/going-infinite)
8. **Hoodwinked** — *John Perkins* (`/hoodwinked`)
   - Access: [Browse Hoodwinked](./index.html#/books/hoodwinked)
9. **How to Listen When Markets Speak** — *Lawrence G. McDonald* (`/how-to-listen-when-markets-speak`)
   - Access: [Browse How to Listen When Markets Speak](./index.html#/books/how-to-listen-when-markets-speak)
10. **How to Make Money in Stocks** — *William J. O&#x27;Neil* (`/how-to-make-money-in-stocks`)
   - Access: [Browse How to Make Money in Stocks](./index.html#/books/how-to-make-money-in-stocks)
11. **Liar&#x27;s Poker** — *Michael Lewis* (`/liars-poker`)
   - Access: [Browse Liar&#x27;s Poker](./index.html#/books/liars-poker)
12. **Lords of Finance** — *Liaquat Ahamed* (`/lords-of-finance`)
   - Access: [Browse Lords of Finance](./index.html#/books/lords-of-finance)
13. **Lying for Money** — *Dan Davies* (`/lying-for-money`)
   - Access: [Browse Lying for Money](./index.html#/books/lying-for-money)
14. **Material World** — *Ed Conway* (`/material-world`)
   - Access: [Browse Material World](./index.html#/books/material-world)
15. **One Up on Wall Street** — *Peter Lynch* (`/one-up-on-wall-street`)
   - Access: [Browse One Up on Wall Street](./index.html#/books/one-up-on-wall-street)
16. **Poor Charlie&#x27;s Almanack** — *Charles T. Munger* (`/poor-charlies-almanack`)
   - Access: [Browse Poor Charlie&#x27;s Almanack](./index.html#/books/poor-charlies-almanack)
17. **Quantitative Momentum** — *Gray, Wesley R.,Vogel, Jack R.* (`/quantitative-momentum`)
   - Access: [Browse Quantitative Momentum](./index.html#/books/quantitative-momentum)
18. **Reminiscences of a Stock Operator** — *Edwin Lefèvre* (`/reminiscences-of-a-stock-operator`)
   - Access: [Browse Reminiscences of a Stock Operator](./index.html#/books/reminiscences-of-a-stock-operator)
19. **Stock Market Wizards** — *Jack D. Schwager* (`/stock-market-wizards`)
   - Access: [Browse Stock Market Wizards](./index.html#/books/stock-market-wizards)
20. **The Ascent of Money** — *Niall Ferguson* (`/the-ascent-of-money`)
   - Access: [Browse The Ascent of Money](./index.html#/books/the-ascent-of-money)
21. **The Big Short** — *Michael Lewis* (`/the-big-short`)
   - Access: [Browse The Big Short](./index.html#/books/the-big-short)
22. **The Bond King** — *Mary Childs* (`/the-bond-king`)
   - Access: [Browse The Bond King](./index.html#/books/the-bond-king)
23. **The Case for Gold** — *Ron Paul* (`/the-case-for-gold`)
   - Access: [Browse The Case for Gold](./index.html#/books/the-case-for-gold)
24. **The Deficit Myth** — *Stephanie Kelton* (`/the-deficit-myth`)
   - Access: [Browse The Deficit Myth](./index.html#/books/the-deficit-myth)
25. **The End of Indexing** — *Niels Joachim Gormsen* (`/the-end-of-indexing`)
   - Access: [Browse The End of Indexing](./index.html#/books/the-end-of-indexing)
26. **The Euro** — *Stiglitz, Joseph E.* (`/the-euro`)
   - Access: [Browse The Euro](./index.html#/books/the-euro)
27. **The Gray Rhino** — *Michele Wucker* (`/the-gray-rhino`)
   - Access: [Browse The Gray Rhino](./index.html#/books/the-gray-rhino)
28. **The Intelligent Option Investor** — *Erik Kobayashi-Solomon* (`/the-intelligent-option-investor`)
   - Access: [Browse The Intelligent Option Investor](./index.html#/books/the-intelligent-option-investor)
29. **The Intelligent REIT Investor Guide** — *Thomas, Brad* (`/the-intelligent-reit-investor-guide`)
   - Access: [Browse The Intelligent REIT Investor Guide](./index.html#/books/the-intelligent-reit-investor-guide)
30. **The Little Book of Common Sense Investing** — *John C. Bogle* (`/the-little-book-of-common-sense-investing`)
   - Access: [Browse The Little Book of Common Sense Investing](./index.html#/books/the-little-book-of-common-sense-investing)
31. **The Little Book That Beats the Market** — *Joel Greenblatt* (`/the-little-book-that-beats-the-market`)
   - Access: [Browse The Little Book That Beats the Market](./index.html#/books/the-little-book-that-beats-the-market)
32. **The Money Culture** — *Michael Lewis* (`/the-money-culture`)
   - Access: [Browse The Money Culture](./index.html#/books/the-money-culture)
33. **The Next Millionaire Next Door** — *Ph. J. D. Stanley* (`/the-next-millionaire-next-door`)
   - Access: [Browse The Next Millionaire Next Door](./index.html#/books/the-next-millionaire-next-door)
34. **The Next Perfect Trade** — *Alex Gurevich* (`/the-next-perfect-trade`)
   - Access: [Browse The Next Perfect Trade](./index.html#/books/the-next-perfect-trade)
35. **The Price of Time** — *Edward Chancellor* (`/the-price-of-time`)
   - Access: [Browse The Price of Time](./index.html#/books/the-price-of-time)
36. **The Psychology of Money** — *Morgan Housel* (`/the-psychology-of-money`)
   - Access: [Browse The Psychology of Money](./index.html#/books/the-psychology-of-money)
37. **The World for Sale** — *Javier Blas and Jack Farchy* (`/the-world-for-sale`)
   - Access: [Browse The World for Sale](./index.html#/books/the-world-for-sale)
38. **Trading Like a Stock Market Wizard** — *Minervini, Mark* (`/trading-like-a-stock-market-wizard`)
   - Access: [Browse Trading Like a Stock Market Wizard](./index.html#/books/trading-like-a-stock-market-wizard)

To convert a rights-approved EPUB into the published `books/` layout, use:

```bash
uv run python scripts/convert_epub.py "E:\ebook\Books\path\book.epub" --slug book-slug
```

For Kindle-format sources (AZW3/AZW/MOBI/KFX) or DJVU (scanned-document) sources, use `scripts/convert_azw3.py`. It transcodes the file to EPUB via Calibre's `ebook-convert`, then compiles the same markdown output — identical to the EPUB path. Requires [Calibre](https://calibre-ebook.com/download) installed:

```bash
uv run python scripts/convert_azw3.py "E:\ebook\Calibre Library\Author\Book (1)\Book - Author.azw3" --slug book-slug
```

---

## 💬 On-Device AI Assistant

The library is a **single-page app**: a hash router (`#/` = home, `#/books/<slug>` = reader) fetches each book's `index.md`, renders it with marked.js, and injects it into a persistent reader view, so the floating **AI** assistant stays open across book switches — its LLM pipeline, embeddings, and conversation history persist in memory. It runs **entirely in the browser** via WebGPU — no question ever leaves the user's device.

- **Top bar** shows a home icon, the book title, and the active chapter number + title (tracked as you scroll).
- **Scope toggle** in the chat panel — *This chapter* / *This book* / *All books* — controls which slice of the corpus the assistant searches for supporting excerpts. The current section's text is always the priority context.
- Powered by **Gemma 4 E2B** (~3.1 GB, downloaded once and cached) with **embeddinggemma-300m** for retrieval.
- Cites other books by title (e.g. `[Book: The Big Short]`) and links to them (routing through the SPA so the chatbot stays open).

### Terminal chatbot (CLI)

Chat with the same book corpus from the command line — no browser required. The CLI uses the **same local Gemma 4 models** as the browser assistant (text-only `Gemma4ForCausalLM`, q4f16). Nothing is sent to external LLM APIs.

**Prerequisites**

```bash
cd scripts && npm install          # once — transformers.js + onnxruntime-node
uv sync --extra cli                # numpy for RAG retrieval
```

First run downloads model weights from Hugging Face (cached locally afterward):

| `--model` | Hugging Face ID | Effective params | Download size |
|-----------|-----------------|------------------|---------------|
| `e2b` (default) | `onnx-community/gemma-4-E2B-it-ONNX` | ~2.3B | ~3.1 GB |
| `e4b` | `onnx-community/gemma-4-E4B-it-ONNX` | ~4.5B | ~6 GB |

**Device selection** (`--device`, default `auto`):

| Value | Behavior |
|-------|----------|
| `auto` | WebGPU when an NVIDIA GPU is detected (same backend as the browser chatbot); otherwise CPU |
| `webgpu` | Force WebGPU in Node |
| `cpu` | Force CPU (slower, always reliable) |
| `dml` | DirectML on Windows — available but unreliable for Gemma 4; prefer `webgpu` |

If GPU inference returns an empty or very short answer, the CLI automatically retries on CPU.

Query embedding (`--embed-device`) stays on CPU by default — the small embedder is fast and WebGPU can crash in Node for that model.

**Examples**

```bash
# One-shot question (E2B, auto GPU)
uv run --extra cli python scripts/chatbot_cli.py -q "What is a black swan?"

# Larger E4B model on GPU
uv run --extra cli python scripts/chatbot_cli.py --model e4b -q "Explain contango"

# Scope to one book or chapter
uv run --extra cli python scripts/chatbot_cli.py --book antifragile --scope book
uv run --extra cli python scripts/chatbot_cli.py --book antifragile --section section-10 --scope chapter

# Show retrieved excerpts before the answer
uv run --extra cli python scripts/chatbot_cli.py --show-context -q "OPEC spare capacity"

# Force CPU
uv run --extra cli python scripts/chatbot_cli.py --device cpu

# Interactive REPL
uv run --extra cli python scripts/chatbot_cli.py
```

**Interactive REPL commands:** `/scope all|book|chapter`, `/book <slug>`, `/section <id>`, `/show-context`, `/quit`.

**Useful flags:** `--top-k`, `--max-new-tokens`, `--no-stream`, `--embed-device`.

See [`.cursor/skills/chatbot-cli/SKILL.md`](.cursor/skills/chatbot-cli/SKILL.md) for agent-oriented usage.

When books are added or converted, regenerate the RAG index and re-wire:

```bash
uv run python scripts/build_chatbot_index.py            # rebuild assets/chatbot_chunks.json (per-section)
cd scripts && node build_chatbot_embeddings.mjs          # rebuild assets/chatbot_embeddings.bin (auto GPU)
cp web_assets/spa.css web_assets/spa.js web_assets/chatbot.css web_assets/chatbot.js assets/
cp web_assets/vendor/marked.esm.js assets/vendor/
uv run python scripts/wire_chatbot.py                   # wire SPA+chatbot into root index.html
```

The embedding build auto-selects **DirectML (GPU)** on Windows when an NVIDIA GPU is present, otherwise CPU — roughly a 10× speedup on a 3060 Ti. Override with `--device cpu|dml|cuda`. The q8 weights yield identical vectors on any device, so GPU and CPU builds are interchangeable.

See [AGENTS.md](./AGENTS.md) for full architecture details.

---

## 🛠️ Repository Ecosystem

The repository is organized as follows:
* **`index.html`** (Root): The SPA shell — hash router, home/reader views, top bar. Serves as the index for GitHub Pages (`github.io`).
* **`books/`**: Published book content as `index.md` per slug (fetched and rendered by the SPA at runtime) plus the catalog metadata.
* **`assets/`**: Served SPA + chatbot assets — `spa.{css,js}`, vendored `marked.esm.js`, `chatbot.{css,js}`, the per-section RAG chunk index, and the prebuilt embedding cache.
* **`web_assets/`**: Source of truth for the SPA/chatbot CSS/JS (copied to `assets/` to publish).
* **`scripts/`**: Contains standalone Python mirroring, EPUB conversion, chatbot index/embedding build, wiring scripts, and the **terminal chatbot CLI** (`chatbot_cli.py`, `chatbot_rag.py`, `embed_query.mjs`).
* **`AGENTS.md`**: Dedicated instructions for AI coding and reading agents detailing the environment, script execution, and structure.
* **`CONTRIBUTING.md`**: Contributing guidelines specifying the automated, agent-led PR workflow.

---

## 🚀 How to Run the Mirrors Locally

To refresh or update the books from their live sources, you can run the mirroring scripts in the `scripts/` folder using Python:

### Prerequisites
Python dependencies (`beautifulsoup4`, `lxml`, `markdownify`) are resolved automatically by [uv](https://docs.astral.sh/uv/) via the project's `pyproject.toml` — no manual install needed. Install uv once:
```bash
pip install uv
```

### Run Scripts
Each script runs independently to fetch and rebuild its respective book:

```bash
# Mirror and rebuild NatGas 101 (with custom SVG charts injection)
uv run python scripts/mirror_natgas101.py

# Mirror and rebuild Oil 101
uv run python scripts/mirror_oil101.py
```

*Note: The scripts make use of `curl.exe` under the hood to fetch materials reliably on Windows environments.*