Pyrolyzer-8894 has made available an HTTP API enabling the use of its carbon calculations infrastructure in the context of CAMEO. We chose to expose the functionality as an HTTP API (not a command-line program) so that we can make updates immediately available.
In order to use this API, you will need:
my-auth-token-890DFGHjfkdlsfJHE45457UHGE76jdfhksfY
.The API is available at https://api-cameo.sig-gis.com/cameo/v1.
The first step as with doing an analysis on a scenario from the CAMEO site consists of uploading the Tree Data, by calling /upload-file/<filename-suffix>
:
import os.path
import requests
import json
import time
# Base API URL
api_url = "https://api-cameo.sig-gis.com/cameo/v1/relay-server"
cameo_auth_token = 'YOUR-CAMEO-AUTH-TOKEN'
user_id = '<YOUR-USER-ID>'
fvs_variant = '<TWO-LETTER-VARIANT>'
def upload_file(file_path: str):
filename = os.path.basename(file_path)
with open(file_path, 'rb') as input_file:
http_resp = requests.post(
f'{api_url}/upload-file/{filename}',
headers={'Authorization': f'Bearer {cameo_auth_token}'},
files={'file': (file_path, input_file)},
timeout=10
)
http_resp.raise_for_status()
logical_path = http_resp.json()
return logical_path
tree_data_logical_path = upload_file("/path/to/tree_data.xls")
cameo_to_fvs_keyfile = upload_file("/path/to/first_keyfile.key")
fvs_keyfile = upload_file("/path/to/second_keyfile.key")
The value returned by the /relay-server/upload-file/:filename_suffix
endpoint is a 'logical path' - something that the API will now how to translate to a concrete file location on the CAMEO cluster. Clients should consider it an opaque value and not try to understand its structure.
To submit the job to help you do the all the process involved in the carbon calculations. We are now ready to actually trigger simulations, by calling /relay-server/submit-job
:
# Only these arguments are technically required arguments - but you are free to change any of them.
def submit_job(
user_id,
fvs_variant,
scenario_name,
fvs_keyfile,
cameo_to_fvs_keyfile,
tree_data_path
):
body = json.dumps(
{
"sig_relay_arguments": {
"scenario-name": "testing scenario",
"fvs-variant": fvs_variant,
"user-id": user_id,
"cameo_to_fvs": {
"keyfile": cameo_to_fvs_keyfile,
"input-spreadsheet": tree_data_path,
"max-dbh": 48,
"max-height": 145,
"model-length": 100,
"num-dbh-breaks": 1,
"num-plot-size": 2,
"num-sub-plot-size": 1,
"scenario-desc": scenario_name,
"top-volume-id": 1,
"version-id": 1,
"broken-top-id": 1,
"phase-id": 1,
"defect-criteria-id": 1,
"edit-perm-id": 1,
"view-perm-id": 2
},
"fvs": {
"keyfile": fvs_keyfile
},
"fvs_to_carbon": {
"analysis-type-id": 1,
"growth-model-id": 1,
"tree-years-id": 1,
"fall-down-percent": 0
}
}
})
print("Sending:", body)
http_resp = requests.post(
f'{api_url}/submit-job',
headers={
'Authorization': f'Bearer {cameo_auth_token}',
'Content-Type': 'application/json',
'Accept': 'application/json'
},
data=body,
timeout=10
)
http_resp.raise_for_status()
return http_resp.json()['sig_relay_deferred_uid']
my_uid_for_polling = submit_job(user_id, fvs_variant, "test scenario", fvs_keyfile, cameo_to_fvs_keyfile, tree_data_logical_path)
The above queues a request for running simulations in CAMEO's computing infrastructure. We do not immediately get a result: instead we get my_uid_for_polling
, with which we will be able to poll the API to monitor completion status (see next section).
Here is a summary of the option keys available to pass.
The upper-most level of the sig_relay_arguments
map has these keys. Each service has its own child object with their own parameters.
Key | Option | Type | Default | Description |
---|---|---|---|---|
scenario-name | String | Random UUID | The name of the scenario to run (optional) | |
fvs-variant | String (2 characters) | Two-letter FVS Variant code | ||
user-id | (Issued by Dev Team) | Integer | SIG-issued user-id | |
cameo-to-fvs | JSON Object | JSON Object | See below | cameo-to-fvs service arguments |
fvs | JSON Object | JSON Object | See below | fvs service arguments |
fvs-to-carbon | JSON Object | JSON Object | See below | fvs-to-carbon service arguments |
The Cameo To FVS microservice captures the functionality of importing tree data and keywords into Cameo to produce FVS input data.
Key | Option | Type | Default | Description |
---|---|---|---|---|
keyfile | Logical path to keyfile | String | Keyfile path | |
input-spreadsheet | String | Path to tree data spreadsheet | ||
max-dbh | Float | 48.0 | Max DBH import param | |
max-height | Float | 145.0 | Max Height import param | |
model-length | Integer | 100 | Model Length import param | |
num-dbh-breaks | Integer | 1 | Num DBH Breaks import param | |
num-plot-size | Integer | 2 | Num Plot Size import param | |
num-sub-plot-size | Integer | 1 | Num Sub Plot Size import param | |
scenario-desc | String | Random UUID | Scenario description | |
top-volume-id | 1: Woodall Formula 2: Cone Formula | Integer | 1 | |
version-id | 1: Private 2: Draft 3: Under Review 4: Final | Integer | 1 | |
broken-top-id | 1: Cone with DIB 2: 1/3s | Integer | 1 | |
phase-id | 1: Feasibility 2: Project 3: MRV 4: Other | Integer | 1 | Phase ID of the scenario |
defect-criteria-id | 1: Merchantable 2: Total Tree | Integer | 1 | |
edit-perm-id | 1: SIG 2: Organization 3: Locked | Integer | 1 | |
view-perm-id | 1: SIG 2: Organization 3: Public | Integer | 1 |
The FVS Service runs the output from Cameo To FVS through FVS with a specified keyfile, uploaded by the user.
Key | Option | Type | Description |
---|---|---|---|
keyfile | Logical path to keyfile | String | Keyfile path, used to run the FVS scenario |
The FVS To Carbon microservice utilizes the results from the FVS microservice to generate a carbon report, downloadable by users in the form of a 7z archive.
Key | Option | Default | Description |
---|---|---|---|
analysis-type-id | 1: Report Date - 1st 2: Project Start | 1 | Analysis Type ID |
growth-model-id | 1: No Interpolation 2: W&K 3: Ameriflux 4: Dodge 5: Friesner, Ray C. | Growth Model ID | |
tree-years-id | 1: All Years 2: Cruise Only | 1 | All Years or Cruise Only |
fall-down-percent | 1 - 100 | Fall down percent |
The following code snippet shows how to await the results of simulations, by polling the /relay-server/check-job-status/:sig_relay_deferred_uid
endpoint every 5 seconds:
def await_cameo_results(sig_relay_deferred_uid: str, polling_interval_s=1):
dfr_status = 'sig_relay_deferred_pending'
results = None
while dfr_status == 'sig_relay_deferred_pending':
http_resp = requests.get(
f'{api_url}/check-job-status/' + sig_relay_deferred_uid,
headers={'Authorization': f'Bearer {cameo_auth_token}'},
timeout=10
)
http_resp.raise_for_status()
body = http_resp.json()
dfr_status = body['sig_relay_deferred_status']
if dfr_status == 'sig_relay_deferred_success':
results = body['sig_relay_deferred_result']
elif dfr_status == 'sig_relay_deferred_error':
raise Exception("Simulations failed!")
else:
time.sleep(polling_interval_s)
return results
my_results = await_cameo_results(my_uid_for_polling, 5)
print(my_results)
# To download output files
def download_file(url, file_name):
with open(file_name, "wb") as downloaded:
http_resp = requests.get(
url,
headers={'Authorization': f'Bearer {cameo_auth_token}'},
timeout=10
)
downloaded.write(http_resp.content)
for key in ["carbon-report", "fvs-file", "input-database"]:
output_base_path = "path/path/path"
key_file_url = my_results[key]
key_file_name = key_file_url.rsplit('/', 1)[-1]
download_file(key_file_url, f"{output_base_path}/{key_file_name}")