Persist Predictions Across Generative Campaigns¶

Predictions are stored permanently. Re-filter, re-rank, or tighten thresholds across runs without a single extra API call.

Postman API Docs

Python SDK Docs

What you'll learn:

How sequence deduplication and cache lookup work under the hood
How to query the DuckDB store directly with arbitrary SQL
How to resume a pipeline after a crash or session restart

export BIOLMAI_TOKEN=your-token-here

Setup¶

import os
from biolmai.pipeline import (
    DataPipeline, DuckDBDataStore,
    ThresholdFilter, RankingFilter,
    ValidAminoAcidFilter, EmbeddingSpec,
    DiversitySamplingFilter,
)

TOKEN = os.environ.get("BIOLMAI_TOKEN", "")
if not TOKEN:
    raise EnvironmentError(
        "Set BIOLMAI_TOKEN before running.\n"
        "Get one at https://biolm.ai/ui/accounts/user-api-tokens/"
    )

The cache in action¶

Run a pipeline, then re-run it. The second run fires zero API calls.

PEPTIDES = [
    "GIGKFLHSAKKFGKAFVGEIMNS",
    "GLFDIIKKIAESF",
    "KLAKLAKKLAKLAK",
    "RRWWRRWWRR",
    "KWKLFKKI",
]

ds = DuckDBDataStore("sdk_demo.duckdb")

pipeline = DataPipeline(sequences=PEPTIDES, datastore=ds, run_id="run_v1", verbose=True)
pipeline.add_prediction("temberture-regression", extractions="prediction", columns="melting_temperature")
pipeline.add_prediction("biolmsol", extractions="solubility_score", columns="solubility")
pipeline.run()
print("Run 1 complete — check the API call count above")

Added stage: PredictionStage('predict_melting_temperature')
Added stage: PredictionStage('predict_solubility', depends_on=['predict_melting_temperature'])

############################################################
# Pipeline: DataPipeline
# Run ID: run_v1
# Initial sequences: 5
# Streaming: ENABLED
############################################################

Execution plan: 2 level(s)
  Level 1: predict_melting_temperature
  Level 2: predict_solubility

============================================================
Stage: predict_melting_temperature
Input: 5 sequences
  Cached: 0/5
  To compute: 5
  Calling temberture-regression.predict...

Completed: 5 sequences in 1 batches (max 5 concurrent)

StageResult(predict_melting_temperature: in=5, out=5, cached=0, computed=5, filtered=0, time=0.6s)
============================================================

============================================================
Stage: predict_solubility
Input: 5 sequences
Depends on: predict_melting_temperature
  Cached: 0/5
  To compute: 5
  Calling biolmsol.predict...

Completed: 5 sequences in 1 batches (max 5 concurrent)

StageResult(predict_solubility: in=5, out=5, cached=0, computed=5, filtered=0, time=0.4s)
============================================================

############################################################
# Pipeline completed in 1.0s
# Final sequences: 5
############################################################

Run 1 complete — check the API call count above

# Re-run identical pipeline — 0 API calls
pipeline2 = DataPipeline(sequences=PEPTIDES, datastore=ds, run_id="run_v1", verbose=True)
pipeline2.add_prediction("temberture-regression", extractions="prediction", columns="melting_temperature")
pipeline2.add_prediction("biolmsol", extractions="solubility_score", columns="solubility")
pipeline2.run()
print("Run 2 complete — all served from cache")

Added stage: PredictionStage('predict_melting_temperature')
Added stage: PredictionStage('predict_solubility', depends_on=['predict_melting_temperature'])

############################################################
# Pipeline: DataPipeline
# Run ID: run_v1
# Initial sequences: 5
# Streaming: ENABLED
############################################################

Execution plan: 2 level(s)
  Level 1: predict_melting_temperature
  Level 2: predict_solubility

============================================================
Stage: predict_melting_temperature
Input: 5 sequences
  Cached: 5/5
  To compute: 0

StageResult(predict_melting_temperature: in=5, out=5, cached=5, computed=0, filtered=0, time=0.0s)
============================================================

============================================================
Stage: predict_solubility
Input: 5 sequences
Depends on: predict_melting_temperature
  Cached: 5/5
  To compute: 0

StageResult(predict_solubility: in=5, out=5, cached=5, computed=0, filtered=0, time=0.0s)
============================================================

############################################################
# Pipeline completed in 0.0s
# Final sequences: 5
############################################################

Run 2 complete — all served from cache

Query the DuckDB store directly¶

The .duckdb file is a standard DuckDB database. You can run arbitrary SQL against it.

# Schema overview
ds.conn.execute("""
    SELECT prediction_type, model_name, COUNT(*) AS n, 
           ROUND(AVG(value), 3) AS mean, ROUND(MIN(value), 3) AS min, ROUND(MAX(value), 3) AS max
    FROM predictions
    GROUP BY prediction_type, model_name
""").df()

# Full prediction history for every sequence
ds.conn.execute("""
    SELECT s.sequence, p.prediction_type, ROUND(p.value, 3) AS value
    FROM sequences s
    JOIN predictions p ON s.sequence_id = p.sequence_id
    ORDER BY s.sequence, p.prediction_type
""").df()

Re-filter without re-predicting¶

Change your filter threshold and re-run — predictions are already cached.

pipeline3 = DataPipeline(sequences=PEPTIDES, datastore=ds, run_id="run_v3_strict", verbose=True)
pipeline3.add_prediction("temberture-regression", extractions="prediction", columns="melting_temperature")
pipeline3.add_prediction("biolmsol", extractions="solubility_score", columns="solubility")
pipeline3.add_filter(ThresholdFilter("melting_temperature", min_value=50.0))  # stricter threshold
pipeline3.run()
pipeline3.summary()

Added stage: PredictionStage('predict_melting_temperature')
Added stage: PredictionStage('predict_solubility', depends_on=['predict_melting_temperature'])
Added stage: FilterStage('filter_2', depends_on=['predict_solubility'])

############################################################
# Pipeline: DataPipeline
# Run ID: run_v3_strict
# Initial sequences: 5
# Streaming: ENABLED
############################################################

Execution plan: 3 level(s)
  Level 1: predict_melting_temperature
  Level 2: predict_solubility
  Level 3: filter_2

============================================================
Stage: predict_melting_temperature
Input: 5 sequences
  Cached: 5/5
  To compute: 0

StageResult(predict_melting_temperature: in=5, out=5, cached=5, computed=0, filtered=0, time=0.0s)
============================================================

[Stage: predict_solubility] (streaming to filter_2)
  Cached: 5/5
  To compute: 0 (streaming)

predict_solubility: processed 5 sequences
  filter_2: 5 passed filter (filtered 0)

############################################################
# Pipeline completed in 0.1s
# Final sequences: 5
############################################################

Cleanup¶

ds.close()
import os; os.remove("sdk_demo.duckdb")

Next Steps¶

Check out additional tutorials at jupyter.biolm.ai, or head over to our BioLM Documentation to explore additional models and functionality.

All Examples

Persist Predictions Across Generative Campaigns¶

Setup¶

The cache in action¶

Query the DuckDB store directly¶

Re-filter without re-predicting¶

Cleanup¶

Next Steps¶

See more use-cases and APIs on your BioLM Console Catalog.¶

BioLM hosts deep learning models and runs inference at scale. You do the science.¶

Contact us to learn more.¶

Accelerate yourLead generation

	prediction_type	model_name	n	mean	min	max
0	solubility	biolmsol	5	1.239	0.368	2.327
1	melting_temperature	temberture-regression	5	67.347	60.310	79.101

	sequence	prediction_type	value
0	GIGKFLHSAKKFGKAFVGEIMNS	melting_temperature	62.521
1	GIGKFLHSAKKFGKAFVGEIMNS	solubility	0.458
2	GLFDIIKKIAESF	melting_temperature	61.510
3	GLFDIIKKIAESF	solubility	0.368
4	KLAKLAKKLAKLAK	melting_temperature	79.101
5	KLAKLAKKLAKLAK	solubility	2.327
6	KWKLFKKI	melting_temperature	73.292
7	KWKLFKKI	solubility	1.766
8	RRWWRRWWRR	melting_temperature	60.310
9	RRWWRRWWRR	solubility	1.276

	Stage	Input	Output	Cached
0	predict_melting_temperature	5	5	5
1	predict_solubility	5	5	0
2	filter_2	5	5	0