init commit

2026-03-06 15:50:39 -07:00
commit 5b279865a1
5 changed files with 413 additions and 0 deletions
@@ -0,0 +1,353 @@
+# LogJensticks - Application Management System
+
+## Project Overview
+LogJensticks is a trucking application management system that automates the processing of job applications through document verification, OCR analysis, and automated checks.
+
+## Architecture
+
+### Infrastructure
+- **2 Docker Containers**: Go server + MongoDB database orchestrated with Docker Compose
+- **Build System**: Makefile for building, running, and testing
+  - `make build` - Compiles code and builds Docker containers
+  - `make run` - Starts all containers
+  - `make run server` - Starts only the server
+  - `make run db` - Starts only MongoDB
+  - `make run test` - Runs unit tests
+
+---
+
+## API Endpoints
+
+### Authentication
+| Method | Endpoint | Access | Description |
+|--------|----------|--------|-------------|
+| POST | `/login` | Public | User login with credentials |
+| POST | `/logout` | Authenticated | User logout |
+| GET | `/health` | Public | Health check endpoint |
+
+> **Unregistered Flow**: Truckers may use the system without registering. Upon first upload/submit the server issues a random `unreg_token` stored in a **secure, HTTP‑only** cookie. "Secure" means the browser will only send it over HTTPS connections, preventing exposure on unencrypted networks. "HTTP‑only" means JavaScript running in the browser cannot read the cookie; this reduces the risk of token theft via XSS. All subsequent requests (save, submit, get, status) require that cookie; the token is a single-use key that only allows access to applications created with the same token. Tokens cannot access others' applications. Registration upgrades the session to a normal authenticated user and allows reuse of prior documents.
+
+### Application Management (Trucker)
+| Method | Endpoint | Access | Description |
+|--------|----------|--------|-------------|
+| POST | `/saveApplication` | Public or cookie | Save application as draft (auto-generates `app_id`). Sets/reads `anon_token` cookie when user is not registered. |
+| POST | `/submitApplication` | Public or cookie | Submit application for review (sets `submitted_timestamp`); requires valid token if unregistered. |
+| GET | `/getApplication/:applicationID` | Authenticated or cookie | Retrieve application details. Registered users may view their whole history; unregistered users may only view apps tied to their current `unreg_token`.  |
+| GET | `/applicationStatus/:applicationID` | Authenticated or cookie | Check status and results with same access rules as above. |
+
+### Application Review (Approver/Manager)
+| Method | Endpoint | Access | Approvers Only | Description |
+|--------|----------|--------|---|-------------|
+| GET | `/approvalQueue` | Yes | - | Get list of pending applications |
+| GET | `/getApplication/:id` | Yes | - | Retrieve application for review (shows all details including check results) |
+| POST | `/approve/:applicationID` | Yes | - | Approve application with optional internal comments |
+| POST | `/reject/:applicationID` | Yes | - | Permanently reject application with internal reason (applicant will not see reason) |
+| POST | `/return/:applicationID` | Yes | - | Return application to applicant for corrections with required feedback message |
+
+### User Management (Admin/Manager)
+| Method | Endpoint | Access | Description |
+|--------|----------|--------|-------------|
+| POST | `/createUser/:username/:token` | Token Required | Create new user (password in request body, token must match DB record) |
+| GET | `/users` | Admin Only | List all users and their roles |
+| PUT | `/updateUser/:username` | Admin Only | Update user details or role |
+
+### Dashboard (Optional)
+| Method | Endpoint | Access | Description |
+|--------|----------|--------|-------------|
+| GET | `/dashboard/stats` | Admin Only | System statistics and metrics |
+
+---
+
+## Frontend Pages
+
+### Public Pages (HTML/CSS/Vue.js)
+- `/login` - User authentication form
+
+### Trucker Pages (Authenticated or Unregistered)
+- `/apply` - Application submission form (multi-step form with file uploads). Works without login; unregistered users are issued a token cookie. Registered users can save and reuse documents.
+- `/applicationConfirmation` - Post-submission confirmation with application ID and notice about token cookie for unregistered users.
+- `/applicationStatus` - Track application progress and view check results. Unregistered users may only access apps created with current token.
+- `/register` - Optional sign-up page for users who want an account to retain history and reuse files.
+
+### Approver/Manager Pages (Authenticated + Authorized)
+- `/approvalQueue` - List of pending applications for review
+- `/reviewApplication/:applicationID` - Detailed application review interface with approve/reject actions
+- `/dashboard` - Manager dashboard with application statistics
+
+---
+
+## Data Schema
+
+### Application Document Fields
+
+#### User-Provided Fields
+- `load_num` (string) - Load number
+- `carrier_name` (string) - Carrier/company name
+- `MC_num` (string) - Motor Carrier number (validated with regex)
+- `driver_name` (string) - Full name of driver
+- `driver_license_num` (string) - License number (numbers only)
+- `truck_plate_num` (string) - License plate number
+- `driver_cell_phone` (string) - Phone number (must be valid format)
+- `truck_num` (string) - Truck identifier
+- `trailer_num` (string) - Trailer identifier
+
+#### Document Uploads (JPEG, PNG, PDF)
+- `driver_with_truck_num` - Picture showing driver, truck, and visible US DOT/MC/KYU numbers
+- `license_front` - Front of driver's license
+- `license_back` - Back of driver's license
+- `apportioned_cab_card` - Apportioned cab card
+
+#### System-Generated Fields (on submission)
+- `_id` (UUID) - MongoDB document ID, same as `app_id`
+- `app_id` (UUID) - Application ID
+- `submitted_timestamp` (UTC timestamp) - When application was submitted
+- `status` (string) - Current status: "draft", "submitted", "processing", "approved", "rejected", "returned", "human_review"
+- `approver_notes` (string, optional) - Internal notes from approver (not visible to applicant)
+- `correction_feedback` (string, optional) - Feedback message sent to applicant when application is returned
+- `unreg_token` (string, optional) – Random token assigned to unregistered users; stored in cookie and used to authorize access to this application
+
+#### Other Metadata
+- `created_by_user` (string, optional) – Username or ID if submitted by a registered user (otherwise absent)
+- `last_modified` (UTC timestamp) – When the record was last updated
+
+#### Check Result Fields (added during processing)
+- `[CheckName]` (string) - One field per check with value: "processing", "pass", "fail", or "human_review"
+
+---
+
+## Application Processing Pipeline
+
+### Submission Flow
+When an application is submitted, it goes through three stages:
+
+1. **Storage** - Application stored in MongoDB with all user-provided fields and documents
+2. **OCR Processing** - Optical Character Recognition extracts data from uploaded document images
+3. **Automated Checks** - Validation checks run to verify application completeness and accuracy
+
+### Document File Storage Structure
+File uploads are stored with metadata:
+```json
+{
+    "file_uploaded": "<<utc timestamp>>",
+    "file_location": "/path/to/file"
+}
+```
+
+For example, during initial submission:
+```json
+{
+    "_id": "<<uuid>>",
+    "app_id": "<<uuid>>",
+    "load_num": "5395746",
+    "carrier_name": "1 Fab Express",
+    "MC_num": "1206193",
+    "driver_name": "Fabiano Caruana",
+    "driver_license_num": 060414478,
+    "truck_plate_num": "C180XZ",
+    "driver_cell_phone": 3859827777,
+    "truck_num": 1234,
+    "trailer_num": 53302,
+    "submitted_timestamp": "<<utc timestamp>>",
+    "status": "processing",
+    "driver_with_truck_num": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file"
+    },
+    "license_front": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file"
+    },
+    "license_back": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file"
+    },
+    "apportioned_cab_card": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file"
+    }
+}
+```
+
+### OCR Processing
+
+OCR data extracted from images is stored within the document field's `ocr_results` object:
+
+```json
+{
+    "driver_with_truck_num": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file",
+        "ocr_results": {
+            "US_DOT": 3418975,
+            "MC_num": "1206193",
+            "KYU": "639224"
+        }
+    },
+    "license_front": {
+        "file_uploaded": "<<utc timestamp>>",
+        "file_location": "/path/to/file"
+    },
+    ...
+}
+```
+
+#### OCR Job Configuration
+Each OCR job is defined with a JSON schema specifying which fields to extract. Fields without a `maps_to` property are stored using their `field_name` as the key:
+
+```json
+{
+    "document_type": "driver_with_truck_num",
+    "fields": [
+        { "field_name": "US DOT", "type": "String", "maps_to": "US_DOT" },
+        { "field_name": "MC", "type": "String", "maps_to": "MC_num" },
+        { "field_name": "KYU", "type": "String" }
+    ]
+}
+```
+
+### Automated Checks System
+
+#### Check Architecture
+- **Check**: Abstract base class with a `run(document)` function
+- **Primary Checks**: Checks with no dependencies; run immediately upon submission in separate Go routines
+- **Dependent Checks**: Checks that depend on other checks completing first; launched as Go routines upon parent check completion
+
+#### Check Lifecycle
+1. Check begins → Set field in document to `"processing"`
+2. Check executes logic
+3. Check completes → Update field to one of: `"pass"`, `"fail"`, or `"human_review"`
+4. Dependent checks are launched if check passed
+5. Human review queue updated if `"human_review"` or `"fail"` result
+
+#### Check Result Format
+Each check adds a field to the document with status:
+```json
+{
+    "app_id": "<<uuid>>",
+    "validate_mc_number": "pass",
+    "validate_phone_number": "pass",
+    "verify_license_valid": "human_review",
+    "cross_reference_us_dot": "fail"
+}
+```
+
+### Application State Visibility & Anti-Gaming Strategy
+
+**Critical Rule**: Applicants must never know WHY their application was rejected unless an approver explicitly chooses to share that information. This prevents applicants from gaming the system through trial-and-error re-submissions.
+
+#### What Applicants CAN See
+- **Approved**: Full approval message
+- **Returned**: Only the `correction_feedback` message from the approver
+- **Processing**: General status ("Your application is being reviewed")
+- **Draft**: Their own unsent application
+
+#### What Applicants CANNOT See
+- **Rejected**: Application status shows "rejected" but NO reason provided
+- Check results and names (even passed checks)
+- Approver internal notes (`approver_notes` field)
+- Specific validation failures or OCR data
+- Which checks failed or passed
+
+#### What Approvers CAN See (Full Visibility)
+- Complete application data
+- All OCR results and extracted data
+- All check results (passed, failed, human_review)
+- Previous approver notes and decision history
+- Full validation details for debugging
+
+#### Workflow States from Applicant Perspective
+- **draft** → (applicant submits) → **submitted** → **processing** → **approved/rejected/returned**
+- "returned" shows feedback for corrections; applicant can resubmit
+- "rejected" shows no details; applicant cannot resubmit unless approver explicitly allows
+- **Unregistered token behavior**: each new unregistered session creates a fresh token. A token cannot access or enumerate previous sessions; clearing cookies removes access. This helps protect sensitive documents from snooping by others using the same browser.
+---
+
+## User Roles & Access Control
+
+### Role Types
+- **Unregistered Trucker** – No login required. Receives a transient `unreg_token` cookie to upload and view only the applications tied to that token. Cannot see previous tokens’ data. Token persists until browser clears it or is upgraded by registration.
+- **Registered Trucker/Applicant** – Logs in normally. Can view full submission history, reuse documents, and update profile.
+- **Approver/Manager** – Can review applications, approve/reject/return, manage users
+- **Admin** – Full system access, user management, configuration
+
+### Authentication
+All endpoints except `/login` and `/health` require authentication via:
+- Session token or JWT (implementation detail)
+- Each authenticated endpoint validates user role
+
+---
+
+## Error Handling & Validation
+
+### Field Validation
+- `MC_num`: Regex pattern validation required
+- `driver_license_num`: Numbers only
+- `driver_cell_phone`: Valid phone number format
+- `truck_num`, `trailer_num`: Numbers only
+- All file uploads: Must be JPEG, PNG, or PDF (max size: [TBD])
+
+### API Response Format (Recommended)
+```json
+{
+    "success": true,
+    "data": { ... },
+    "error": null
+}
+```
+
+Or on error:
+```json
+{
+    "success": false,
+    "data": null,
+    "error": {
+        "code": "VALIDATION_ERROR",
+        "message": "Invalid MC number format"
+    }
+}
+```
+
+### HTTP Status Codes
+- `200 OK` - Successful request
+- `201 Created` - Resource created
+- `400 Bad Request` - Invalid input or missing/invalid token for unregistered user
+- `401 Unauthorized` - Missing/invalid authentication
+- `403 Forbidden` - Insufficient permissions
+- `404 Not Found` - Resource not found (including token mismatch)
+- `500 Internal Server Error` - Server error
+
+---
+
+## Database Considerations
+
+### Recommended MongoDB Indexes
+- `_id` (automatic)
+- `submitted_timestamp` (for sorting/filtering applications)
+- `status` (for querying by application status)
+- `driver_license_num` (for duplicate detection)
+- `app_id` (if not using `_id` as primary key)
+
+### Data Retention
+- Applications should be archived after [TBD] days/months
+- Document files may need separate storage (S3/filesystem)
+
+---
+
+## Security Considerations
+
+- **Token-based User Creation**: `/createUser` requires a valid onboarding token to prevent unauthorized user creation
+- **Role-based Access**: All protected endpoints must verify user role before granting access
+- **Password Storage**: Passwords should be hashed (bcrypt/Argon2 recommended)
+- **File Uploads**: Validate file types and scan for malware
+- **OCR Accuracy**: Limit OCR confidence thresholds; flag low-confidence results for human review
+
+---
+
+- Email notifications when application status changes
+- Application history/audit trail
+- Batch processing for legacy applications
+- Admin dashboard with analytics and reporting
+- Integration with external validation services (FMCSA, state registries)
+- Multi-language support
+- Mobile application
+- Webhook integrations for downstream systems