# LogJensticks - Application Management System

## Project Overview
LogJensticks is a trucking application management system that automates the processing of job applications through document verification, OCR analysis, and automated checks.

## Architecture

### Infrastructure
- **3 Docker Containers**: Caddy (reverse proxy + static frontend) + Go server + MongoDB, orchestrated with Docker Compose
- **Build System**: Makefile for building, running, and testing
  - `make build` - Compiles Go binary
  - `make run` - Builds Docker images and starts all containers
  - `make run-server` - Starts only the Go server
  - `make run-db` - Starts only MongoDB
  - `make run-caddy` - Starts only Caddy
  - `make run-frontend` - Starts Vite dev server locally (no Docker)
  - `make run-test` - Runs unit tests

---

## API Endpoints

### Authentication
| Method | Endpoint | Access | Description |
|--------|----------|--------|-------------|
| POST | `/login` | Public | User login with credentials |
| POST | `/logout` | Authenticated | User logout |
| GET | `/health` | Public | Health check endpoint |

> **Unregistered Flow**: Carriers may use the system without registering. Upon first upload/submit the server issues a random `unreg_token` stored in a **secure, HTTP‑only** cookie. "Secure" means the browser will only send it over HTTPS connections, preventing exposure on unencrypted networks. "HTTP‑only" means JavaScript running in the browser cannot read the cookie; this reduces the risk of token theft via XSS. All subsequent requests (save, submit, get, status) require that cookie; the token is a single-use key that only allows access to applications created with the same token. Tokens cannot access others' applications. Registration upgrades the session to a normal authenticated user and allows reuse of prior documents.

### Application Management (Carrier)
| Method | Endpoint | Access | Description |
|--------|----------|--------|-------------|
| POST | `/saveApplication` | Public or cookie | Save application as draft (auto-generates `app_id`). Sets/reads `anon_token` cookie when user is not registered. |
| POST | `/submitApplication` | Public or cookie | Submit application for review (sets `submitted_timestamp`); requires valid token if unregistered. |
| GET | `/getApplication/:applicationID` | Authenticated or cookie | Retrieve application details. Registered users may view their whole history; unregistered users may only view apps tied to their current `unreg_token`.  |
| GET | `/applicationStatus/:applicationID` | Authenticated or cookie | Check status and results with same access rules as above. |

### Application Review (Broker)
| Method | Endpoint | Access | Brokers Only | Description |
|--------|----------|--------|---|-------------|
| GET | `/approvalQueue` | Yes | - | Get list of pending applications |
| GET | `/getApplication/:id` | Yes | - | Retrieve application for review (shows all details including check results) |
| POST | `/approve/:applicationID` | Yes | - | Approve application with optional internal comments |
| POST | `/reject/:applicationID` | Yes | - | Permanently reject application with internal reason (carrier will not see reason) |
| POST | `/return/:applicationID` | Yes | - | Return application to carrier for corrections with required feedback message |

### Lane Management
| Method | Endpoint | Access | Description |
|--------|----------|--------|-------------|
| POST | `/lanes` | Broker | Create a new lane; returns the lane ID and shareable link |
| GET | `/lanes` | Broker | List all lanes created by the authenticated broker |
| GET | `/lanes/:laneID` | Public (via link) | View lane details |
| POST | `/lanes/:laneID/bid` | Authenticated Carrier | Place a bid on a lane; adds carrier's ID to `bidding_carriers` |

### User Management (Admin/Manager)
| Method | Endpoint | Access | Description |
|--------|----------|--------|-------------|
| POST | `/createUser/:username/:token` | Token Required | Create new user (password in request body, token must match DB record) |
| GET | `/users` | Admin Only | List all users and their roles |
| PUT | `/updateUser/:username` | Admin Only | Update user details or role |

### Dashboard (Optional)
| Method | Endpoint | Access | Description |
|--------|----------|--------|-------------|
| GET | `/dashboard/stats` | Admin Only | System statistics and metrics |

---

## Frontend Pages

### Public Pages (HTML/CSS/Vue.js)
- `/login` - User authentication form

### Carrier Pages (Authenticated or Unregistered)
- `/apply` - Application submission form (multi-step form with file uploads). Works without login; unregistered users are issued a token cookie. Registered users can save and reuse documents.
- `/applicationConfirmation` - Post-submission confirmation with application ID and notice about token cookie for unregistered users.
- `/applicationStatus` - Track application progress and view check results. Unregistered users may only access apps created with current token.
- `/register` - Optional sign-up page for users who want an account to retain history and reuse files.

### Broker Pages (Authenticated + Authorized)
- `/approvalQueue` - List of pending applications for review
- `/reviewApplication/:applicationID` - Detailed application review interface with approve/reject actions
- `/dashboard` - Manager dashboard with application statistics
- `/lanes` - List of broker's lanes with bid counts
- `/lanes/create` - Form to create a new lane
- `/lanes/:laneID` - Lane detail view showing all bids

### Carrier Pages (Lane Bidding)
- `/lanes/:laneID` - Public lane view; authenticated carriers can place a bid

---

## Data Schema

### Application Document Fields

#### User-Provided Fields
- `load_num` (string) - Load number
- `carrier_name` (string) - Carrier/company name
- `MC_num` (string) - Motor Carrier number (validated with regex)
- `driver_name` (string) - Full name of driver
- `driver_license_num` (string) - License number (numbers only)
- `truck_plate_num` (string) - License plate number
- `driver_cell_phone` (string) - Phone number (must be valid format)
- `truck_num` (string) - Truck identifier
- `trailer_num` (string) - Trailer identifier

#### Document Uploads (JPEG, PNG, PDF)
- `driver_with_truck_num` - Picture showing driver, truck, and visible US DOT/MC/KYU numbers
- `license_front` - Front of driver's license
- `license_back` - Back of driver's license
- `apportioned_cab_card` - Apportioned cab card

#### System-Generated Fields (on submission)
- `_id` (UUID) - MongoDB document ID, same as `app_id`
- `app_id` (UUID) - Application ID
- `submitted_timestamp` (UTC timestamp) - When application was submitted
- `status` (string) - Current status: "draft", "submitted", "processing", "approved", "rejected", "returned", "human_review"
- `broker_notes` (string, optional) - Internal notes from broker (not visible to carrier)
- `correction_feedback` (string, optional) - Feedback message sent to carrier when application is returned
- `unreg_token` (string, optional) – Random token assigned to unregistered users; stored in cookie and used to authorize access to this application

#### Other Metadata
- `created_by_user` (string, optional) – Username or ID if submitted by a registered user (otherwise absent)
- `last_modified` (UTC timestamp) – When the record was last updated

#### Check Result Fields (added during processing)
- `[CheckName]` (string) - One field per check with value: "processing", "pass", "fail", or "human_review"

### Lane Document Fields

- `_id` (UUID) - MongoDB auto-generated document ID; used in URLs and shareable links
- `lane_id` (string, nullable) - Brokerage's own internal reference number for the lane; provided by the broker, not generated by the system. May be null until the broker assigns one, but must be filled in eventually.
- `pickup_state` (string, nullable) - Two-letter state code for pickup location (e.g. `"CA"`); may be derived from `pickup_address`
- `dropoff_state` (string, nullable) - Two-letter state code for dropoff location; may be derived from `dropoff_address`
- `pickup_address` (string, nullable) - Full pickup address
- `dropoff_address` (string, nullable) - Full dropoff address
- `bidding_carriers` (array of strings) - `_id`s of carriers who have placed a bid
- `created_by` (string) - Username of the broker who created the lane
- `created_by_id` (ObjectID) - MongoDB `_id` of the broker user document

> **Validation rule**: At least one of the following must be provided at creation time: `lane_id`, both state codes, or both addresses. A lane cannot be created with none of these.

#### Example Lane Document
```json
{
    "_id": "<<mongo uuid>>",
    "lane_id": "LN-2094",
    "pickup_state": "CA",
    "dropoff_state": "TX",
    "pickup_address": "123 Warehouse Blvd, Los Angeles, CA 90001",
    "dropoff_address": "456 Distribution Dr, Houston, TX 77001",
    "bidding_carriers": ["<<carrier _id>>", "<<carrier _id>>"],
    "created_by": "broker_jane",
    "created_by_id": "<<ObjectID>>"
}
```

---

## Application Processing Pipeline

### Submission Flow
When an application is submitted, it goes through three stages:

1. **Storage** - Application stored in MongoDB with all user-provided fields and documents
2. **OCR Processing** - Optical Character Recognition extracts data from uploaded document images
3. **Automated Checks** - Validation checks run to verify application completeness and accuracy

### Document File Storage Structure
File uploads are stored with metadata:
```json
{
    "file_uploaded": "<<utc timestamp>>",
    "file_location": "/path/to/file"
}
```

For example, during initial submission:
```json
{
    "_id": "<<uuid>>",
    "app_id": "<<uuid>>",
    "load_num": "5395746",
    "carrier_name": "1 Fab Express",
    "MC_num": "1206193",
    "driver_name": "Fabiano Caruana",
    "driver_license_num": 060414478,
    "truck_plate_num": "C180XZ",
    "driver_cell_phone": 3859827777,
    "truck_num": 1234,
    "trailer_num": 53302,
    "submitted_timestamp": "<<utc timestamp>>",
    "status": "processing",
    "driver_with_truck_num": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file"
    },
    "license_front": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file"
    },
    "license_back": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file"
    },
    "apportioned_cab_card": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file"
    }
}
```

### OCR Processing

OCR data extracted from images is stored within the document field's `ocr_results` object:

```json
{
    "driver_with_truck_num": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file",
        "ocr_results": {
            "US_DOT": 3418975,
            "MC_num": "1206193",
            "KYU": "639224"
        }
    },
    "license_front": {
        "file_uploaded": "<<utc timestamp>>",
        "file_location": "/path/to/file"
    },
    ...
}
```

#### OCR Job Configuration
Each OCR job is defined with a JSON schema specifying which fields to extract. Fields without a `maps_to` property are stored using their `field_name` as the key:

```json
{
    "document_type": "driver_with_truck_num",
    "fields": [
        { "field_name": "US DOT", "type": "String", "maps_to": "US_DOT" },
        { "field_name": "MC", "type": "String", "maps_to": "MC_num" },
        { "field_name": "KYU", "type": "String" }
    ]
}
```

### Automated Checks System

#### Check Architecture
- **Check**: Abstract base class with a `run(document)` function
- **Primary Checks**: Checks with no dependencies; run immediately upon submission in separate Go routines
- **Dependent Checks**: Checks that depend on other checks completing first; launched as Go routines upon parent check completion

#### Check Lifecycle
1. Check begins → Set field in document to `"processing"`
2. Check executes logic
3. Check completes → Update field to one of: `"pass"`, `"fail"`, or `"human_review"`
4. Dependent checks are launched if check passed
5. Human review queue updated if `"human_review"` or `"fail"` result

#### Check Result Format
Each check adds a field to the document with status:
```json
{
    "app_id": "<<uuid>>",
    "validate_mc_number": "pass",
    "validate_phone_number": "pass",
    "verify_license_valid": "human_review",
    "cross_reference_us_dot": "fail"
}
```

### Application State Visibility & Anti-Gaming Strategy

**Critical Rule**: Carriers must never know WHY their application was rejected unless a broker explicitly chooses to share that information. This prevents carriers from gaming the system through trial-and-error re-submissions.

#### What Carriers CAN See
- **Approved**: Full approval message
- **Returned**: Only the `correction_feedback` message from the broker
- **Processing**: General status ("Your application is being reviewed")
- **Draft**: Their own unsent application

#### What Carriers CANNOT See
- **Rejected**: Application status shows "rejected" but NO reason provided
- Check results and names (even passed checks)
- Broker internal notes (`broker_notes` field)
- Specific validation failures or OCR data
- Which checks failed or passed

#### What Brokers CAN See (Full Visibility)
- Complete application data
- All OCR results and extracted data
- All check results (passed, failed, human_review)
- Previous broker notes and decision history
- Full validation details for debugging

#### Workflow States from Carrier Perspective
- **draft** → (carrier submits) → **submitted** → **processing** → **approved/rejected/returned**
- "returned" shows feedback for corrections; carrier can resubmit
- "rejected" shows no details; carrier cannot resubmit unless broker explicitly allows
- **Unregistered token behavior**: each new unregistered session creates a fresh token. A token cannot access or enumerate previous sessions; clearing cookies removes access. This helps protect sensitive documents from snooping by others using the same browser.
---

## User Roles & Access Control

### Role Types
- **Unregistered Carrier** – No login required. Receives a transient `unreg_token` cookie to upload and view only the applications tied to that token. Cannot see previous tokens’ data. Token persists until browser clears it or is upgraded by registration.
- **Registered Carrier** – Logs in normally. Can view full submission history, reuse documents, and update profile.
- **Broker** – Can review applications, approve/reject/return, manage users
- **Admin** – Full system access, user management, configuration

### Authentication
All endpoints except `/login` and `/health` require authentication via:
- Session token or JWT (implementation detail)
- Each authenticated endpoint validates user role

---

## Error Handling & Validation

### Field Validation
- `MC_num`: Regex pattern validation required
- `driver_license_num`: Numbers only
- `driver_cell_phone`: Valid phone number format
- `truck_num`, `trailer_num`: Numbers only
- All file uploads: Must be JPEG, PNG, or PDF (max size: [TBD])

### API Response Format (Recommended)
```json
{
    "success": true,
    "data": { ... },
    "error": null
}
```

Or on error:
```json
{
    "success": false,
    "data": null,
    "error": {
        "code": "VALIDATION_ERROR",
        "message": "Invalid MC number format"
    }
}
```

### HTTP Status Codes
- `200 OK` - Successful request
- `201 Created` - Resource created
- `400 Bad Request` - Invalid input or missing/invalid token for unregistered user
- `401 Unauthorized` - Missing/invalid authentication
- `403 Forbidden` - Insufficient permissions
- `404 Not Found` - Resource not found (including token mismatch)
- `500 Internal Server Error` - Server error

---

## Database Considerations

### Recommended MongoDB Indexes

**applications collection**
- `_id` (automatic)
- `submitted_timestamp` (for sorting/filtering)
- `status` (for querying by status)
- `driver_license_num` (for duplicate detection)

**lanes collection**
- `_id` (automatic)
- `created_by` (for listing a broker's lanes)

**users collection**
- `username` (unique)

**sessions collection**
- `token` (unique)
- `expires_at` (TTL)

### Data Retention
- Applications should be archived after [TBD] days/months
- Document files may need separate storage (S3/filesystem)

---

## Security Considerations

- **Token-based User Creation**: `/createUser` requires a valid onboarding token to prevent unauthorized user creation
- **Role-based Access**: All protected endpoints must verify user role before granting access
- **Password Storage**: Passwords should be hashed (bcrypt/Argon2 recommended)
- **File Uploads**: Validate file types and scan for malware
- **OCR Accuracy**: Limit OCR confidence thresholds; flag low-confidence results for human review

---

- Email notifications when application status changes
- Application history/audit trail
- Batch processing for legacy applications
- Admin dashboard with analytics and reporting
- Integration with external validation services (FMCSA, state registries)
- Multi-language support
- Mobile application
- Webhook integrations for downstream systems