RAG with newer 30B models, Google Pinpoint and VLMs
Previously, I shared experience of trying out LightRAG to see if it was accurate and useful enough to help speed up some workflows. It had promise, but not as good as I had hoped for initially. There was some mix ups in context of different documents, and it often missed details considered critical in order for it to be useful for me. As with anything AI related these days, things are updated almost on a weekly basis, and now a few months later things have improved considerably.
LightRAG
LightRAG has improved a lot. It's updated almost on a weekly basis. Processing documents can take a long time on commodity hardware, and when there are crashes or interruptions, it didn't recover gracefully and documents had to reprocessed from the start. This is no longer the case. I'm a bit more confident in it now to add processing new documents as part of my workflow for future queries.
While the Qwen3-8B model was quite fast as it fit into the 16GB of VRAM I have available, it missed details and gets confused with context of different issues. These issues seems to be addressed when using newer and larger Qwen3-30B-A3B-Thinking-2507 model.
A much larger LLM model, also means I'll need to customise inference server more. It's going to be very slow with mix of GPU and CPU, as my GPU only has 16GB of memory and 30B models will use like 20-30GB of memory. Ollama is something that just works out of the box, which is great for most things, but not for specific workloads. For RAG processing I opted to use llama.cpp where it's much easier to tweak options.
Currently I'm using Unsloth's 4bit quantised version of Qwen3-30B with the following llama.cpp parameters. Which gives me around ~20 tokens per second. This is acceptable, as waiting a minute or two for detailed response, beats going through hundreds of pages of documents.
# LLM server
llama-server --n-cpu-moe 24 --n-gpu-layers 32 -t 16 --ctx-size 32768 --port 8000 --jinja -fa on -m /storage/AI/models/Qwen3-30B-A3B-Thinking-2507-Q4_K_M.gguf
# Reranker
llama-server --ubatch-size 2048 -ngl 0 --reranking --port 8001 -m /storage/AI/models/bge-reranker-v2-m3-Q8_0.gguf
The test case I had was if the LLM was good enough to figure out a financial note detail in some stock exchange filings as well as context of a company annual report.
TLDR; Yes it it was good enough.
Petaling Jaya Dispersal Link Expressway (PJD)
The test case, was the controversial PJD Highway. It was an unpopular highway proposal that despite multiple protests, cancellations and promises by politicians not to be built, kept on being proposed by government agencies.
First to ask a generic question: What do we know about PJD Link?
The response was quite good, we can see here that a large engineering company is facing financial difficulties. And as part of it's proposal, was planning to acquire a company that is about to get.
Also it referenced only the relevant documents. I have dozens more unrelated documents currently in the same RAG database.
### Understanding PJD Link EntitiesPJD Link refers to two distinct entities within a business context involving a proposed acquisition by Scomi Energy Services Bhd (SESB). The primary entities are PJD Link Holdings Sdn Bhd and PJD Link (M) Sdn Bhd, which played specific roles in a Framework Agreement.
PJD Link (M) Sdn Bhd served as the Target Company in a proposed acquisition by SESB. The Target Company was described as being "principally involved in developing, designing, constructing, operating and managing highway construction projects" in SESB's documentation. [^1]
PJD Link Holdings Sdn Bhd functioned as a key party to the Framework Agreement with SESB, alongside Noblemax Resources Sdn Bhd. This entity was described as "a party involved in the Framework Agreement for the proposed acquisition of Target Company" and was identified as "one of the parties to the Framework Agreement with SESB for the proposed acquisition." [^1]
### Framework Agreement and Termination
The Framework Agreement was executed on 3 October 2022 between SESB, PJD Link Holdings Sdn Bhd, Noblemax Resources Sdn Bhd, and PJD Link (M) Sdn Bhd (the Target Company). This agreement was intended to consolidate the parties' intentions for SESB's proposed acquisition of 100% equity interest in the Target Company, which SESB planned to implement as part of its regularisation plan. [^2]
The Framework Agreement was mutually terminated on 17 July 2023 via a Notice of Mutual Termination. This termination occurred after Bursa Malaysia Securities Berhad rejected SESB's application for an extension of time for submitting its regularisation plan on 13 July 2023. SESB then reviewed its Proposed Acquisition and decided not to pursue it, leading to the mutual termination of the Framework Agreement with PJD Link Holdings Sdn Bhd and Noblemax Resources Sdn Bhd. [^1]
### Key Details of the Proposed Acquisition
The proposed acquisition was intended to be part of SESB's rehabilitation plan to uplift its PN17 status. SESB had initially entered into the Framework Agreement with a proposed indicative purchase consideration of RM922.00 million. As part of the terms of the Framework Agreement, SESB had paid a refundable deposit of RM12.0 million to the PJD Vendors (which included PJD Link Holdings Sdn Bhd and PJD Link (M) Sdn Bhd), which was secured by a third-party legal charge over a land asset in India. [^2]
### Summary of Relationship
The relationship between SESB and PJD entities was formalized through the Framework Agreement, which was meant to facilitate a proposed acquisition. However, the acquisition process was terminated on 17 July 2023, and SESB has since pursued other business development opportunities to create new revenue streams and support its rehabilitation plan.
### References
- [1] scomi-annual-report-2023_1755495673673_0.md
- [2] 2024-01-11-SCOMI-Announcement_1755495736955_0.md
We get a struggling public company, that is about to be de-listed, is acquiring a company that is planning to build a highway.
How it planned to acquire the company was dubious, because it had no cash to cover the proposed RM 1 Billion expected value of PJD link. What we see is a refundable RM 12 million "deposit" backed by some property in India.
We find out that the asset wasn't really worth RM12 million, SESB only got RM800K back, and still trying to sell the asset, which is only valued RM7 million.
What was this land asset in India?
### The Land Asset in India: The Charged Asset
The land asset in India referred to in the documents is known as the "Charged Asset." It is a piece of land measuring approximately 4,678 square feet, including a house and premises, located at Door No. 35/1, New No. 77, Poes Garden, Chennai 600086, India.^[^2]
This land asset was used as collateral for a deposit paid by Scomi Energy Services Bhd (SESB) in the context of a Framework Agreement. Specifically, as part of the Framework Agreement signed on 3 October 2022, SESB paid a refundable deposit of RM12.0 million to the PJD Vendors (PJD Link Holdings Sdn Bhd and Noblemax Resources Sdn Bhd), which was secured by a third-party legal charge over this land asset.^[^2]
The Framework Agreement was mutually terminated on 17 July 2023, leading to the full impairment of the deposit in SESB's audited financial statements for the year ended 30 June 2023. From 1 July 2023 up to LPD (latest practicable date), RM0.80 million of the deposit was repaid.^[^2] SESB's Board is currently pursuing the sale of the Charged Asset to recover the remaining deposit, with an expectation to realize the asset at a gross disposal consideration of USD1.50 million (approximately RM6.963 million based on the exchange rate of RM4.6420:USD 1.00 as at LPD).^[^2]
The Charged Asset is specifically mentioned in the financial statement disclosures and is part of the geographical segments where SESB operates, with India listed in the revenue breakdown table showing revenue contributions.^[^1]
### References
- [1] scomi-annual-report-2023_1755495673673_0.md
- [2] 2024-01-11-SCOMI-Announcement_1755495736955_0.md
Some caveats
As I shared in my previous note, that AI augments, but doesn't replace your other skills and tools. You're going to need to find the documents of course, and know what to look for in these documents. The Investigative Tipsheet for Malaysia is helpful for this.
While in this case we have all the information needed provided by AI, you should scan through the key documents yourself. What I do is scan through it, and quickly jot notes down on things that stand out, to be investigated further, and questions I need to ask.
You also need to understand what's going on with the subject matter, and if not, ask a human who is. You shouldn't depend on AI.
For skills in investigating finance stories, I highly recommend joining Finance Uncovered courses, and the book "The Investigative Journalist's guide to Company Accounts" published by the Centre for Investigative Journalism, UK.
Google Pinpoint
While LightRAG provides great Local AI tool summaries with large context. It only provides references to the source documents. It can be quite tedious to find that document and then search through for the specific text of section to verify.
Google Pinpoint fills this gap. It can quickly organise a lot of documents and manually or automatically label them. Automatically organise them by date, entities and other categorisations automatically. Then be able to search and jump straight to verify an answer as highlighted text in a document, when you need quick verifications to questions and summaries.
I find it works especially well for short documents like news articles, parliamentary replies and inquiries and legal documents.
VLM plugins and scripts for photo and video libraries
The idea to use Visual Language Models (VLMs) to describe, search video for specific clips including timecodes has been used on some pretty impressive plugins for DaVinci Resolve. On a simpler level, it really should also be applicable for photo libraries. I use Darktable and for some time have been manually tagging and describing photos. This can now also be done by VLMs, except it's a bit tedious to cut and paste the results, even though it is much faster than before. I came across Ayeda Okambawa's LUA plugin to do similar things for Darktable. It's not exactly how I envision an AI plugin would work, it but it has given me inspiration to spend a future Sunday morning coding a plugin to quickly fill in description field with Getty Image style description and suggested keywords.
Note this is for generic descriptions, for documentation work, I have to write, proper caption and description with complete context. Still it saves a lot of time to generate keywords automatically and to be able to do quick searches like "coastal road", "bus stop", "Kuala Lumpur city skyline, sunset"
Suggest photo description in style of Getty Images. Suggest keywords in one line. Provide answers in plain text without titles.
A freight train carrying cement cars travels alongside a dense, verdant jungle backdrop featuring a towering limestone karst formation under a partly cloudy sky. The train is lined with brightly colored cars bearing the “YTL Cement” logo. The scene evokes a sense of industrial transportation amidst a dramatic natural landscape.
Keywords: train, cement, transport, jungle, karst, limestone, industrial, freight, YTL Cement, landscape, natural, transport, construction.
Document Actions