A headless setup runs LM Studio with no desktop app and no window. You control it with commands only. You need a headless setup on a computer that has no screen, such as a Linux computer that you reach over a network. This vignette also covers a second case: R runs on your computer, and LM Studio runs on another one.
vignette("getting-started") covers the steps that are
the same with or without the desktop app: the server, the model list,
the download, the load, and the chat. This vignette covers only what
differs. Read getting-started first, but skip its install
step, which opens a web browser. The next section replaces it.
LM Studio comes with a command-line tool called lms.
Without the desktop app, you install lms and the rest of LM
Studio from the command line.
install_lmstudio(method = "headless") runs the LM Studio
install script for your system. If lms is already
installed, at version 0.4.0 or later, the function installs nothing.
The install writes files to your computer, so the function does not
install without your consent. In R at the console, it asks you first,
and it installs only if you answer yes. When R runs a script with no
console, for example with Rscript, R cannot ask you. There,
the function stops with an error, unless you set the environment
variable RLMSTUDIO_ALLOW_INSTALL to "true"
before the call. An environment variable is a named setting that
programs read when they run. In R,
Sys.setenv(RLMSTUDIO_ALLOW_INSTALL = "true") sets it.
# Install LM Studio without the desktop app. At the console, it asks you first.
rlmstudio::install_lmstudio(method = "headless")After the install, restart R. Then load the package and check that R
can find lms.
library(rlmstudio)
# TRUE if R finds the lms tool
has_lms()
#> [1] TRUE
# TRUE if the lms tool is version 0.4.0 or later
check_lms_version()
#> ✔ LM Studio CLI is using the modern architecture (0.4.0+).
#> [1] TRUEIf has_lms() returns FALSE, find the full
path of lms on your computer. Then set the environment
variable RLMSTUDIO_LMS_PATH to that path, and the package
uses it.
A daemon is a program that runs in the background, with no window.
The LM Studio daemon is called llmster. When the desktop
app is open, the app does the work of the daemon. Without the desktop
app, start the daemon yourself, before you start the server.
# Start the LM Studio daemon in the background
lms_daemon_start()
#> ✔ LM Studio daemon started in the background.lms_daemon_status() returns the lines that the command
lms status prints, as text. They say, for example, whether
the server is on. Where the output tells you to run
lms server start in a terminal, the R function
lms_server_start() of the next section runs that command
for you.
You start the server as in getting-started.
lms_server_start() waits until LM Studio answers, for about
wait seconds. If the wait runs out, the function gives a
warning, and your script goes on. On a computer that is slow to start
the server, give a longer wait. Then call
lms_server_ready(), which returns TRUE only
when LM Studio answers.
# Start the server, and wait about 60 seconds for it to answer
lms_server_start(wait = 60)
#> ✔ LM Studio server started successfully on the default port.
# TRUE if LM Studio answers
lms_server_ready()
#> [1] TRUEIf the wait runs out, or if lms_server_ready() returns
FALSE, LM Studio did not answer. One cause is that it needs
more time. Another is that another program holds the port. A port is a
number that picks one program on a computer, and LM Studio uses port
1234 by default. Another cause is that the server turned the request
away. A server turns a request away when it requires an API token, a
secret string that it checks with each request, and the request carries
none. The next section shows how to send one.
Now download, load, and chat with a model as
getting-started shows. The calls are the same without the
desktop app.
An API token works like a password for the server. A server that requires one turns away each request that does not carry it. If you use a server that someone else runs, get the token from that person. The LM Studio documentation on authentication shows how to turn on the token check and create tokens in the desktop app.
The package reads the token from the environment variable
RLMSTUDIO_API_TOKEN. To set it for every R session, put a
line such as RLMSTUDIO_API_TOKEN=your-token in the file
.Renviron in your home folder, and restart R. R reads that
file when it starts. If the file does not exist, create it. Then each
function that sends a request to the server sends the token with it.
You can also pass a token to one call with the token
argument. Every function that sends a request to the server has this
argument. A token argument wins over the environment
variable, so use it when one script talks to two servers with different
tokens. The call below passes the token from the environment
variable.
# Pass the token to one call
lms_server_ready(token = Sys.getenv("RLMSTUDIO_API_TOKEN"))
#> [1] TRUEDo not type the token itself into your script. An error message in R can repeat the line of code that failed, and the token would then show on screen. Read it from the environment variable, as above.
If the server turns a request away, the error message says so. If you
sent no token, the message tells you to set
RLMSTUDIO_API_TOKEN or to pass token. If you
sent a token, the message says that the server rejected it.
When you are done for now, stop the server.
LM Studio can run on another computer, such as one with a large
graphics card, while you run R on your laptop. The host is the computer
that runs LM Studio. The host argument gives its address,
such as "http://192.168.1.20:1234". That is the network
address of the host, then a colon and the port of the server. The
package asks port 1234 by default. By default, host is
"http://localhost:1234", which is your own computer. The
person who manages the host can tell you its network address.
The functions that run lms, such as
lms_daemon_start() and lms_server_start(), act
on the computer where R runs. So run them in R on the host. By default,
the server answers only requests that come from the host itself. To let
other computers reach it, set the environment variable
LMS_SERVER_HOST to "0.0.0.0" before you start
the server. The address 0.0.0.0 means every network address
of the host.
# On the host: let other computers reach the server
Sys.setenv(LMS_SERVER_HOST = "0.0.0.0")
lms_server_start(wait = 60)Then any computer that can reach the host over the network can send requests to the server. If the network is not private, require an API token on the server. The LM Studio documentation shows how to turn on the token check only in the desktop app. On a host without the desktop app, keep the server on a private network that only computers you trust can reach. A firewall between the two computers can block the port.
Your own computer needs the package, but not LM Studio. Pass the
address of the host to each call with the host argument.
Every function that sends a request to the server has this argument, for
example lms_load(), lms_chat(), and
lms_unload(). The package then sends your prompts to that
computer. The model reads them there, and the replies come back to
R.
A script that runs on its own, for example every night, must start
the daemon and stop it again. with_lms_daemon() does both.
It starts the daemon, runs your code, and then stops the server and the
daemon. It stops them even if your code fails with an error. It returns
the value of your code.
Take care on a computer that other people use. At the end,
with_lms_daemon() stops the server and the daemon even if
they ran before the call, so it can stop a server that someone else
uses. If the desktop app is open, the daemon keeps running, as the next
section says.
When you no longer need LM Studio, stop the daemon with
lms_daemon_stop(). It returns TRUE when the
daemon stopped or was not running. Use force = TRUE to stop
the server first.
If the desktop app is open, lms_daemon_stop() does not
stop the daemon. The function returns FALSE and prints a
message that says so.