S3000 REST webservice support

While our commercial framework for laboratories, S3000, had support for making predictions via webservices for a long time, that was limited to asynchronous ones: a webservice endpoint receives data coming in and, once the predictions have been generated, the results get forwarded to another webservice.

With recent changes to the codebase, it is now possible to offer synchronous REST webservices as well. In order to reduce latency as much as possible, provenance logging under the hood has been modified to have a much higher throughput that no longer impacts the speed of the predictions.

Thanks to the plugin architecture of S3000, customer-specific webservices can be implemented and deployed with minimal effort.

image-dataset-converter release

A new release of our image-dataset-converter-all library is now available: 0.0.11. Docker images have been deployed as well.

The most notably changes since 0.0.7 are:

  • support for placeholders is now available for readers/writers, which can be used in constructing input/output files/folders, including predefined ones available ({CWD}, {HOME}, {TMP}), input-based ones (e.g., {INPUT_PATH}, {INPUT_NAMEEXT}), user-defined ones (supplied to tools, e.g., via the -p/--placeholders option of the idc-convert tool) and run-time ones (set with the set-placeholder filter)

  • added the --resume_from option to applicable readers, which allows resuming the pipeline from the file matching the supplied glob, e.g., */012345.jpg

  • the new from-multi reader and to-multi writer simplify the combining of datasets (from potentially different formats) and output in multiple formats respectively

  • writers that can split the incoming stream into subsets had the new --split_group option added, which allows keeping samples together within subsets using a regular expression, e.g., when dealing with images that were split into sub-grids or augmented with flipping/rotating

SpeciesNet 4.0.1 Docker images available

First Docker images are available for the SpeciesNet network that Google announced on March 3rd, 2025:

Below is an example on how to use these images (on Linux or on Windows under WSL2).

Prerequisites:

  • create a directory for your output eg "speciesnet"

  • in that directory create the following sub-directories

    • cache

    • config

    • data

    • output

Processing data:

  • copy the images that you want to analyze into the "speciesnet/data" directory

  • from the "speciesnet" directory launch the appropriate docker image in interactive mode

    • CPU:

      docker run --rm --gpus=all --shm-size 8G --net=host \
  -u $(id -u):$(id -g) -e USER=$USER \
  -v `pwd`:/workspace \
  -v `pwd`/cache:/.cache \
  -v `pwd`/config:/.config \
  -v `pwd`/cache:/.torch \
  -it waikatodatamining/speciesnet:4.0.1_cpu

    • CUDA:

      docker run --rm --gpus=all --shm-size 8G --net=host \
  -u $(id -u):$(id -g) -e USER=$USER \
  -v `pwd`:/workspace \
  -v `pwd`/cache:/.cache \
  -v `pwd`/config:/.config \
  -v `pwd`/cache:/.torch \
  -it waikatodatamining/speciesnet:4.0.1_cuda21.1

  • run the following script to process your images:

    speciesnet_run_model \
    --folders "/workspace/data" \
    --predictions_json "/workspace/output/predictions.json"

Or, if you want to run the individual steps separately:

speciesnet_run_model --detector_only \
    --folders "/workspace/data" \
    --predictions_json "/workspace/output/detections.json"
speciesnet_run_model --classifier_only \
    --folders "/workspace/data" \
    --detections_json "/workspace/output/detections.json" \
    --predictions_json "/workspace/output/classifications.json"
speciesnet_run_model --ensemble_only \
    --folders "/workspace/data" \
    --detections_json "/workspace/output/detections.json" \
    --classifications_json "/workspace/output/classifications.json" \
    --predictions_json "/workspace/output/predictions.json"

On your host system, the "speciesnet/output" directory will then contain the generated .json file(s), with "predictions.json" containing all the relevant information (classification and bbox).

For more information on the json output format:

https://github.com/google/cameratrapai/tree/main?tab=readme-ov-file#output-format

S3000 Python support

Our commercial offering for laboratories, S3000, now has official support for custom Python models. Being a Java application, S3000 historically relied on Weka for its modeling. However, with the integration of Jep (Java Embedded Python) in ADAMS, it is now possible to use scikit-learn or deep learning libraries for training models and making predictions.

In order to make this work, ADAMS needs to launch from the context of an activated virtual Python environment that contains all the required Python libraries and then use the JepRegressor Weka meta-classifier to define the relevant code for training and making predictions. Due to Java serialization not being applicable to Python objects, these scripts must implement the relevant serialization/deserialization themselves, e.g., via pickle.

image-dataset-converter release

A new release of our image-dataset-converter-all library is now available: 0.0.7. Docker images have been deployed as well.

The most notably changes are:

image-dataset-converter release

A new release of our image-dataset-converter-all library is now available: 0.0.6. Docker images have been deployed as well.

The most notably changes are:

  • using 90% as default quality for JPEG images now as a good compromise between quality and size (PIL uses 75% by default), this can be overridden with environment variable IDC_JPEG_QUALITY

  • added methods to idc.api module: jpeg_quality(), array_to_image(...), empty_image(...)

  • numpy is limited to <2.0.0 to avoid issues with the imgaug library