VibeOps

Unlocking the Power of Nomic for Effective Embeddings

In the expansive world of document processing, nomic stands as a pivotal tool when it comes to embedding content effectively. Its use is not just a technical necessity but a strategic choice that can enhance the way we interact with and understand documents.

A Deep Dive into Embeddings

What Are Embeddings?
Embeddings are numerical representations of text that capture the contextual meaning of words, phrases, or entire documents. They become especially powerful when dealing with multiple documents, allowing for advanced comparisons, clustering, and information retrieval.

Why Nomic for Embeddings?
Nomic is specifically designed to facilitate the embedding process, making it a preferred choice for developers and researchers alike. Using nomic ensures that the integrity and essence of your content are preserved while boosting efficiency in models requiring semantic understanding.

Token Limitations

When utilizing embeddings, be mindful of the token limitations inherent in many processing systems. Each document should remain below 8,192 tokens. This cap is a vital parameter that allows the LLm to process your document

So try to follow the below cookbook

• embed the documents
• embed the query
• find the most relevant document wth a similarity search implemented in python
• call smollm with the relevant document passed as a context
• I pasted below some untested ai generated code that should do the similarity calculation and call smollm, I initially staretd in powershell but makes sense to create and end to end python script, will keep u posted

import json
import numpy as np
import requests

# load embedding
with open("embedding_doc1.json") as f:
    doc1_emb = np.array(json.load(f)["data"][0]["embedding"])
with open("embedding_doc2.json") as f:
    doc2_emb = np.array(json.load(f)["data"][0]["embedding"])
with open("embedding_question.json") as f:
    query_emb = np.array(json.load(f)["data"][0]["embedding"])

# cosine function similarity
def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

sim1 = cosine_similarity(doc1_emb, query_emb)
sim2 = cosine_similarity(doc2_emb, query_emb)

if sim1 > sim2:
    chosen = "doc1.txt"
else:
    chosen = "doc2.txt"

with open("question.txt", encoding="utf8") as f:
    domanda = f.read()
with open(chosen, encoding="utf8") as f:
    contesto = f.read()

# Prompt for SmolLM2
prompt = f"Domanda: {domanda}\nContesto:\n{contesto}"

# call SmolLM2 
res = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "smollm2:135m",
        "prompt": prompt,
        "stream": False
    }
)
print(res.json()["response"])