Skip to content

Commit 0eefd2f

Browse files
npuchois-aphpnpuchois-aphp
authored
feat: add local development documentation (#21)
1 parent 4e752ea commit 0eefd2f

9 files changed

Lines changed: 353 additions & 11 deletions

File tree

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,3 +55,4 @@ solr_auth*
5555
postgresql.jar
5656
log/
5757
cre.env
58+
.env

‎.local/README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Cohort360-QueryExecutor Local Helper Repository
2+
3+
This directory contains a refactored and cleaner set of scripts for local development, integrated with the root project's existing Docker configuration.
4+
5+
## Features
6+
- **Integrated**: Uses the root `Dockerfile` and `docker-compose.yml`.
7+
- **Colored Output**: Scripts use a utility logger for better readability.
8+
- **Flexible**: Supports both direct Server Mode (running JAR) and Docker Mode.
9+
- **Isolated**: Keeps local configurations and temporary files out of the main repository's git history.
10+
11+
## Table of Contents
12+
1. [Prerequisites](#prerequisites)
13+
2. [Configuration](#configuration)
14+
3. [Setup](#setup)
15+
4. [Running Server Mode (Direct JAR)](#running-server-mode-direct-jar)
16+
5. [Running Docker Mode](#running-docker-mode)
17+
6. [API Usage](#api-usage)
18+
19+
---
20+
21+
## Prerequisites
22+
- **Java 11 or 17** (for Server Mode)
23+
- **Docker and Docker Compose** (for Docker Mode)
24+
- **Maven** (handled via `./mvnw` wrapper)
25+
26+
---
27+
28+
## Configuration
29+
Before running anything, initialize your local environment:
30+
31+
```bash
32+
cp .local/env.example .local/.env
33+
```
34+
35+
Edit `.local/.env` to configure your local environment (FHIR URL, PostgreSQL, Solr, etc.).
36+
37+
---
38+
39+
## Setup
40+
To build the project and download required dependencies (like the PostgreSQL driver):
41+
42+
```bash
43+
./.local/scripts/setup.sh
44+
```
45+
46+
This script:
47+
1. Builds the project using Maven.
48+
2. Downloads the PostgreSQL driver JAR.
49+
3. Automatically initializes `.local/.env` if it doesn't exist.
50+
51+
---
52+
53+
## Running Server Mode (Direct JAR)
54+
Runs the application directly on your host machine. This is faster for iterative development.
55+
56+
```bash
57+
./.local/scripts/run-server.sh
58+
```
59+
60+
This script:
61+
- Loads environment variables from `.local/.env`.
62+
- Generates `solr_auth.txt` for Solr authentication.
63+
- Applies the necessary JVM `--add-opens` flags (matching the production `entrypoint.sh`).
64+
65+
---
66+
67+
## Running Docker Mode
68+
Runs everything in containers.
69+
70+
```bash
71+
./.local/docker/run-docker.sh
72+
```
73+
74+
This script:
75+
- Builds the `sjs:latest` image using the root `Dockerfile`.
76+
- Starts services using the root `docker-compose.yml` with local overrides (`.local/docker/docker-compose.local.yml`).
77+
- Includes a local PostgreSQL container for testing if needed.
78+
79+
---
80+
81+
## API Usage
82+
83+
Once the server is running (port `8091` by default):
84+
85+
### Example Query (Count)
86+
```bash
87+
curl -X POST http://localhost:8091/jobs -H "Content-Type: application/json" -d '{
88+
"input": {
89+
"cohortDefinitionSyntax": "{\"sourcePopulation\":{\"caresiteCohortList\":[118]},\"_type\":\"request\",\"request\":{\"_type\":\"andGroup\",\"_id\":0,\"isInclusive\":true,\"criteria\":[{\"_type\":\"basicResource\",\"_id\":1,\"isInclusive\":true,\"resourceType\":\"Patient\",\"filterFhir\":\"active=true&gender=female\",\"criteria\":[],\"dateRangeList\":[],\"temporalConstraints\":[]}],\"dateRangeList\":[],\"temporalConstraints\":[]},\"temporalConstraints\":[]}",
90+
"mode": "count"
91+
}
92+
}'
93+
```
94+
95+
### Check Job Status
96+
```bash
97+
curl http://localhost:8091/jobs
98+
```
99+
100+
### Cancel Job
101+
```bash
102+
curl -X DELETE http://localhost:8091/jobs/<jobId>
103+
```

‎.local/docker/docker-compose.local.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
services:
2+
app:
3+
build:
4+
dockerfile: Dockerfile
5+
container_name: sjs-app-local
6+
environment:
7+
- PG_HOST=db
8+
- SPARK_MASTER=spark://spark-master:7077
9+
ports:
10+
- 8091:8091
11+
env_file:
12+
- .env
13+
14+
spark-master:
15+
container_name: spark-master-local
16+
17+
worker-1:
18+
container_name: spark-worker-1-local
19+
20+
worker-2:
21+
container_name: spark-worker-2-local
22+
23+
# Optional: PostgreSQL for local development
24+
db:
25+
image: postgres:15
26+
container_name: sjs-db-local
27+
environment:
28+
POSTGRES_DB: omop
29+
POSTGRES_USER: postgres
30+
POSTGRES_PASSWORD: postgres
31+
ports:
32+
- 5555:5555

‎.local/docker/run-docker.sh‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/bin/bash
2+
3+
# Change directory to project root
4+
SCRIPT_DIR="$(dirname "$0")"
5+
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
6+
cd "$PROJECT_ROOT"
7+
8+
# Load utils
9+
source .local/scripts/utils.sh
10+
11+
log_step "Checking environment configuration..."
12+
ENV_FILE=".env"
13+
14+
if [ ! -f "$ENV_FILE" ]; then
15+
log_warn "$ENV_FILE file not found. Creating it from .local/env.example..."
16+
cp .local/env.example "$ENV_FILE"
17+
fi
18+
19+
log_step "Starting Docker Compose services..."
20+
sudo docker compose -f docker-compose.yml -f .local/docker/docker-compose.local.yml up --build

‎.local/env.example‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# FHIR Server Configuration
2+
FHIR_URL=http://localhost:8080
3+
FHIR_URL_COHORT=http://localhost:8080
4+
# FHIR_ACCESS_TOKEN=
5+
6+
# Django API Configuration (for callbacks)
7+
DJANGO_CALLBACK_URL=http://localhost:8085
8+
SJS_TOKEN={GENERATED_TOKEN_USE_IN_DJANGO}
9+
10+
# PostgreSQL Configuration
11+
PG_HOST=localhost
12+
PG_PORT=5432
13+
PG_DB=omop
14+
PG_SCHEMA=public
15+
PG_USER=postgres
16+
DB_OMOP_PASSWORD=postgres
17+
18+
# Solr Configuration
19+
SOLR_ZK=localhost:9983
20+
SOLR_USER=
21+
SOLR_PASSWORD=
22+
23+
# Spark Configuration
24+
SPARK_MASTER="local[*]"
25+
SPARK_DRIVER_HOST=localhost
26+
SPARK_DRIVER_PORT=4000
27+
# SPARK_EXECUTOR_MEMORY=2g
28+
29+
# Application Configuration
30+
SJS_APP_PORT=8091
31+
JOBS_THREADS=20
32+
JOBS_AUTO_RETRY=0
33+
USE_SOURCE_POPULATION=true
34+
USE_SOURCE_POPULATION_ON_PATIENT=true
35+
USE_ACTIVE_FILTER=true
36+
COHORT_CREATION_LIMIT=500
37+
DEFAULT_RESOLVER=solr
38+
DEFAULT_COHORT_CREATION_SERVICE=pg

‎.local/scripts/run-server.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/bin/bash
2+
3+
# Change directory to project root
4+
SCRIPT_DIR="$(dirname "$0")"
5+
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
6+
cd "$PROJECT_ROOT"
7+
8+
# Load utils
9+
source .local/scripts/utils.sh
10+
11+
log_step "Loading environment configuration..."
12+
ENV_FILE=".env"
13+
load_env "$ENV_FILE"
14+
15+
# Generate solr_auth.txt if SOLR_USER and SOLR_PASSWORD are set
16+
if [ ! -z "$SOLR_USER" ] && [ ! -z "$SOLR_PASSWORD" ]; then
17+
log_info "Generating solr_auth.txt..."
18+
echo "httpBasicAuthUser=$SOLR_USER" > solr_auth.txt
19+
echo "httpBasicAuthPassword=$SOLR_PASSWORD" >> solr_auth.txt
20+
fi
21+
22+
export JAVA_SOLR_OPT="-Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory -Dsolr.httpclient.config=solr_auth.txt"
23+
24+
# JVM options for Spark 3.4+ and Java 11/17 (mirrors entrypoint.sh)
25+
export JAVA_OPTS="--add-opens=java.base/java.lang=ALL-UNNAMED \
26+
--add-opens=java.base/java.lang.invoke=ALL-UNNAMED \
27+
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED \
28+
--add-opens=java.base/java.io=ALL-UNNAMED \
29+
--add-opens=java.base/java.net=ALL-UNNAMED \
30+
--add-opens=java.base/java.nio=ALL-UNNAMED \
31+
--add-opens=java.base/java.util=ALL-UNNAMED \
32+
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED \
33+
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED \
34+
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
35+
--add-opens=java.base/sun.nio.cs=ALL-UNNAMED \
36+
--add-opens=java.base/sun.security.action=ALL-UNNAMED \
37+
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED \
38+
--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED"
39+
40+
log_success "Starting Cohort Requester Server Mode..."
41+
java $JAVA_OPTS $JAVA_SOLR_OPT -jar target/cohort-requester.jar

‎.local/scripts/setup.sh‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/bin/bash
2+
set -e
3+
4+
# Change directory to project root
5+
SCRIPT_DIR="$(dirname "$0")"
6+
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
7+
cd "$PROJECT_ROOT"
8+
9+
# Load utils
10+
source .local/scripts/utils.sh
11+
12+
log_step "Starting project setup..."
13+
14+
# Check for .env file
15+
if [ ! -f ".local/.env" ]; then
16+
log_warn ".local/.env file not found. Creating it from .local/env.example..."
17+
cp .local/env.example .local/.env
18+
log_info "Please review and edit .local/.env as needed."
19+
fi
20+
21+
log_step "Building the project..."
22+
if ./mvnw clean package -DskipTests; then
23+
log_success "Project built successfully."
24+
else
25+
log_error "Build failed."
26+
exit 1
27+
fi
28+
29+
log_step "Preparing dependencies..."
30+
mkdir -p target
31+
32+
# Fetch PostgreSQL JAR if not present
33+
POSTGRES_VERSION=$(./mvnw help:evaluate -Dexpression=postgres.version -q -DforceStdout)
34+
if [ ! -f "postgresql.jar" ]; then
35+
log_info "Downloading PostgreSQL driver (version $POSTGRES_VERSION)..."
36+
./mvnw org.apache.maven.plugins:maven-dependency-plugin:3.1.1:get -Dartifact=org.postgresql:postgresql:$POSTGRES_VERSION
37+
./mvnw org.apache.maven.plugins:maven-dependency-plugin:3.1.1:copy -Dartifact=org.postgresql:postgresql:$POSTGRES_VERSION -DoutputDirectory=./
38+
mv postgresql-$POSTGRES_VERSION.jar postgresql.jar
39+
log_success "PostgreSQL driver downloaded."
40+
else
41+
log_info "PostgreSQL driver already exists."
42+
fi
43+
44+
log_success "Setup complete!"

‎.local/scripts/utils.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/bin/bash
2+
3+
# Colors
4+
RED='\033[0;31m'
5+
GREEN='\033[0;32m'
6+
YELLOW='\033[0;33m'
7+
BLUE='\033[0;34m'
8+
NC='\033[0m' # No Color
9+
10+
function log_info() {
11+
echo -e "${BLUE}[INFO]${NC} $1"
12+
}
13+
14+
function log_success() {
15+
echo -e "${GREEN}[SUCCESS]${NC} $1"
16+
}
17+
18+
function log_warn() {
19+
echo -e "${YELLOW}[WARN]${NC} $1"
20+
}
21+
22+
function log_error() {
23+
echo -e "${RED}[ERROR]${NC} $1"
24+
}
25+
26+
function log_step() {
27+
echo -e "${BLUE}==>${NC} $1"
28+
}
29+
30+
# Function to load environment variables from a file if it exists
31+
function load_env() {
32+
local env_file="$1"
33+
if [ -f "$env_file" ]; then
34+
log_info "Loading environment variables from $env_file..."
35+
set -a
36+
source "$env_file"
37+
set +a
38+
else
39+
log_warn "Environment file $env_file not found."
40+
fi
41+
}

0 commit comments

Comments
 (0)