Repository navigation
Expand file tree
/
Copy pathembedder.h
More file actions
102 lines (82 loc) · 4.2 KB
/
Copy pathembedder.h
File metadata and controls
102 lines (82 loc) · 4.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
/**
* Embedder — generate embeddings via ONNX Runtime (internal) or a remote
* OpenAI-compatible /v1/embeddings endpoint (external).
*
* Internal mode mirrors ragger_memory/embedding.py from the Python version.
* Uses ONNX models from ~/.ragger/models/ + HuggingFace tokenizer.
*
* External mode sends text to a remote endpoint and reads the returned vector.
*/
#pragma once
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "embed_provider.h"
namespace ragger {
class Embedder : public IEmbedProvider {
public:
/// Disabled mode. Constructs successfully but cannot embed: ready() is
/// false, dimensions() is 0, and encode() returns an empty vector.
///
/// This exists so a bad embedding-model configuration degrades instead of
/// killing the daemon. Ragger keeps accepting and recording conversations;
/// the rows are stored with a NULL embedding and the housekeeping backfill
/// picks them up once the configuration is corrected. Callers that write
/// embeddings must check ready() and defer when it is false — never store
/// the empty vector, which would encode as a zero-length blob and read
/// back as a real (but meaningless) embedding.
Embedder();
/// Internal (ONNX) mode: construct with path to model directory
/// (containing model.onnx + tokenizer.json).
explicit Embedder(const std::string& model_dir);
/// External mode: construct with endpoint details. Embeddings are
/// computed by calling POST /v1/embeddings on the remote server.
Embedder(const std::string& host, int port,
const std::string& model_name,
const std::string& api_key = "",
int dimensions = 0);
~Embedder() override;
/// Encode text to a normalized embedding vector. Returns an empty
/// vector on failure (disabled instance, or — external mode — a
/// request error).
std::vector<float> encode(const std::string& text) const;
/// Embedding dimensions. For internal mode, read from config/model.
/// For external mode, determined by probing or from constructor arg.
int dimensions() const override;
/// True if this embedder can actually produce embeddings. False only for
/// the disabled instance built by the default constructor.
bool ready() const override;
// --- IEmbedProvider adapter ---------------------------------------
// Thin wrappers over encode(): translate its empty-vector-on-failure
// convention into IEmbedProvider's std::optional-on-failure contract.
/// Embed one text; std::nullopt if !ready() or encode() fails.
std::optional<std::vector<float>> embed(const std::string& text) const override;
/// Embed many texts sequentially (Embedder has no subprocess pool to
/// bound, unlike EmbedExecutor — batching here is just a loop).
std::vector<std::optional<std::vector<float>>>
embed_batch(const std::vector<std::string>& texts) const override;
/// True if this embedder uses an external endpoint.
bool is_external() const;
/// (External only) Query the remote endpoint's /v1/models and return
/// available model names. Returns empty vector on error or if internal.
std::vector<std::string> list_remote_models() const;
/// (External only) Probe the endpoint by sending a test embedding
/// request. Returns the vector dimensionality, or 0 on failure.
int probe_dimensions() const;
/// (External only) The model name the server reported in its last
/// embedding response. May differ from the requested model when the
/// endpoint (llama-swap, LM Studio) serves whatever is loaded.
std::string last_served_model() const;
/// (External only) Whether the last probe_dimensions() call returned a
/// plausible embedding vector (finite values, non-constant/non-zero).
/// A wrong model — e.g. a chat LLM misidentified as an embedder and
/// asked to serve /v1/embeddings — can return a fixed-size but
/// meaningless vector (all zeros, or NaN/Inf) instead of erroring, so
/// dimension count alone isn't sufficient to trust the response.
bool last_probe_sane() const;
private:
struct Impl;
std::unique_ptr<Impl> pImpl;
};
} // namespace ragger