vemem — visual entity memory

vemem — visual entity memory

Before you install — what this skill touches on your system

Read this first. This skill handles biometric data, so transparency up front beats surprises later.

What this skill is, exactly

The skill itself is instruction-only. It's the markdown you're reading — no scripts, no executables, no automatic installation of anything. Adding this skill to your ClawHub / Claude Code / Hermes / OpenClaw install does not by itself run code on your machine.

The skill *instructs the agent* to install and use the vemem Python package separately. That package is the component that actually reads images and writes to disk.

If you install the vemem package, here is everything it does

Local state it creates or reads:

Network activity vemem itself produces:

What vemem does NOT do on its own:

The OpenClaw automatic-processing concern is a separate opt-in

The ClawHub review correctly flagged that vemem has a first-party OpenClaw integration that can auto-process every image attachment. That integration is a separate install (vemem-openclaw-sidecar + registering a specific OpenClaw plugin) and is NOT enabled by adding this skill or by installing the base vemem package.

Enable it only if you understand you're granting an always-on face-recognition layer over every image your agent sees. Disable at any time by stopping the sidecar process.

Verification & provenance

Compliance context

vemem stores biometric identifiers. If you deploy it to users other than yourself, YOU are the data controller under GDPR / BIPA / CCPA. The library provides primitives (forget / restrict / export) but does not enforce consent capture — that's your app's responsibility. Full deployer checklist: COMPLIANCE.md.

Recommended first-run posture

1. Install into a dedicated venv, not your system Python. 2. Use a test VEMEM_HOME path (e.g. /tmp/vemem-test) for your first session so you can inspect + delete the store wholesale. 3. Use a local VLM/LLM (Ollama) for the first integration test, not remote APIs, to confirm no images leave your machine. 4. Enable the OpenClaw sidecar integration only after you've seen vemem behave as a manually-invoked tool.

What this skill does

vemem is the identity layer that sits between a vision model (face/object detector) and a text LLM. It turns "an image of a person" into a named, stable entity ID — same person across sessions, same face across angles, same object across lighting.

It keeps track of facts, events, and relationships per entity — like Mem0 or mem0-style stores, but keyed on visual identity rather than on a user_id you have to know in advance.

When to activate

Activate this skill when the user:

Do NOT activate for:

Setup

Quick check — is vemem available?

Run python -c "import vemem; print(vemem.__version__)". If that fails, install:

pip install vemem
or with uv:
uv pip install vemem

First-time face encoding triggers a ~200MB InsightFace model download into ~/.insightface/. Warn the user if their network is constrained.

Run the MCP server (preferred for agents)

python -m vemem.mcp_server

This exposes 14 tools over stdio. Wire it into your host's MCP config. For Claude Desktop, the ready-to-paste config lives at docs/examples/claude_desktop_config.json in the vemem repo.

Use directly (preferred for scripting in Python)

from vemem import Vemem
vem = Vemem()  # LanceDB store at ~/.vemem, InsightFace encoder

Store path is overridable via VEMEM_HOME env var or Vemem(home="/path/to/store").

Core operations — the mental model

There are 13 operations. The ones you'll use most:

Writing identity into the store

| Op | When | |---|---| | observe(image_bytes) | A new image came in. Detect faces/objects, embed, persist. Returns a list of Observations, each with a stable content-hash id. | | label(observation_ids, name) | The user just told you who someone is. Creates the entity if new, binds those observations to it. This is the moment identity becomes permanent. | | remember(entity_id, fact) | Attach a fact to a known entity — "Charlie runs marathons", "the red mug lives in the kitchen". |

Reading identity out

| Op | When | |---|---| | identify(image_bytes, k=5) | Return candidate entities matching the image, ranked by similarity. Each candidate already includes attached facts — you don't need a separate recall call. | | recall(entity_id) | All known facts, events, and relationships for an entity. Use when the user references someone by name. |

Correcting mistakes

| Op | When | |---|---| | relabel(observation_id, new_name) | "That's not Charlie, that's Dana" — reassigns the observation and emits a negative binding so the clusterer won't re-attach it. | | merge(entity_ids) | Two entities turn out to be the same person. Folds them together, preserving facts with provenance. | | split(entity_id, groups) | One entity turns out to be two people. Separates them with cross-wise negative bindings. | | forget(entity_id) | Privacy delete — hard-removes observations, embeddings, bindings, facts. Physically prunes from storage version history (GDPR Art. 17 compliant). Not reversible. | | undo(event_id=None) | Undoes the most recent reversible op by you (within a 30-day window). Does not work on forget. |

Common patterns

Pattern A: camera frame comes in

observations = vem.observe(image_bytes)
candidates = vem.identify(image_bytes, k=3, min_confidence=0.4)
if candidates:
    names = [f"{c.entity.name} (conf {c.confidence:.2f})" for c in candidates]
    print(f"Visible: {', '.join(names)}")
else:
    print(f"Unknown face(s) detected: {len(observations)}. Label with vem.label(obs_ids, name=...).")

Pattern B: user says "that's Charlie"

observations = vem.observe(image_bytes)
charlie = vem.label([o.id for o in observations], name="Charlie")
vem.remember(charlie.id, "we met at the coffee shop on 2026-04-17")

Pattern C: agent needs context for a reply

candidates = vem.identify(image_bytes, k=3)
context_parts = []
for c in candidates:
    fact_strs = "; ".join(f.content for f in c.facts)
    context_parts.append(f"{c.entity.name} (conf {c.confidence:.2f}): {fact_strs}")
context = "People visible: " + " | ".join(context_parts) if context_parts else "No known faces."
Feed context into your LLM's system message or context block.

Pattern D: correction

# identify() said "Charlie" at 0.71 confidence, but user says it's Dana
candidates = vem.identify(image_bytes)
wrong_obs_id = candidates[0].matched_observation_ids[0]
vem.relabel(wrong_obs_id, "Dana")
A negative binding against Charlie is now recorded — the clusterer won't re-assign.

Pattern E: privacy request

# User says "forget everything about Sarah"
sarah = vem.store.find_entity_by_name("Sarah")
if sarah is not None:
    counts = vem.forget(sarah.id)
    print(f"Deleted: {counts}")
    # Pruned from LanceDB version history — actually gone, GDPR-compliant.
    # This is NOT reversible. Warn the user before calling.

Important constraints

Compliance note

vemem stores biometric identifiers. If the host app is deployed to users, the deployer is the data controller under GDPR / BIPA / CCPA. Key primitives:

Full checklist: COMPLIANCE.md in the vemem repo.

Not this skill's job

Troubleshooting

| Symptom | Cause | Fix | |---|---|---| | identify() returns [] on a face you labeled earlier | Different encoder version, or the face isn't being detected | Check encoder.id hasn't changed; try min_confidence=0.2 to see raw scores | | RuntimeError: image pipeline unavailable | InsightFace weights not installed | First call downloads ~200MB from InsightFace to ~/.insightface/; ensure network access on first use | | ModalityMismatchError on merge | Trying to merge a face entity with an object entity | v0 keeps modalities separate; create an instance_of relationship instead | | OperationNotReversibleError on undo | Past 30 days, or op was forget | Not fixable — forget is deliberately irreversible; window is configurable via DEFAULT_UNDO_WINDOW |

Deeper references (bundled with this skill)

Loaded on demand when the agent needs the detail — keep them out of the hot context path.

Upstream references (in the vemem repo)

Repo: https://github.com/linville-charlie/vemem

Quick check — is vemem available?

Run python -c "import vemem; print(vemem.__version__)". If that fails, install:

pip install vemem
or with uv:
uv pip install vemem

First-time face encoding triggers a ~200MB InsightFace model download into ~/.insightface/. Warn the user if their network is constrained.

Run the MCP server (preferred for agents)

python -m vemem.mcp_server

This exposes 14 tools over stdio. Wire it into your host's MCP config. For Claude Desktop, the ready-to-paste config lives at docs/examples/claude_desktop_config.json in the vemem repo.

Use directly (preferred for scripting in Python)

from vemem import Vemem
vem = Vemem()  # LanceDB store at ~/.vemem, InsightFace encoder

Store path is overridable via VEMEM_HOME env var or Vemem(home="/path/to/store").

| Symptom | Cause | Fix | |---|---|---| | identify() returns [] on a face you labeled earlier | Different encoder version, or the face isn't being detected | Check encoder.id hasn't changed; try min_confidence=0.2 to see raw scores | | RuntimeError: image pipeline unavailable | InsightFace weights not installed | First call downloads ~200MB from InsightFace to ~/.insightface/; ensure network access on first use | | ModalityMismatchError on merge | Trying to merge a face entity with an object entity | v0 keeps modalities separate; create an instance_of relationship instead | | OperationNotReversibleError on undo | Past 30 days, or op was forget | Not fixable — forget is deliberately irreversible; window is configurable via DEFAULT_UNDO_WINDOW |