Object Detection Without Python (A Pure R Pipeline with YOLO26)

Wed, 16 Sep 2026 00:00:00 +0000

Introduction

In an earlier post, we ran a pre-trained YOLO model from R by reaching into Python with reticulate: import torch and ultralytics, hand a conda environment to use_condaenv(), and call the model as if it were an R object. That works, but it means every student needs a working Python environment somewhere reachable from R — a conda install, a mapped network drive, an environment.yml — before they can detect a single car.

This post shows the alternative: doing the entire inference step in R, with no Python at runtime at all. It relies on the R torch package, which binds directly to libtorch (the C++ library that PyTorch itself is built on). Once a model has been exported to the TorchScript format, torch::jit_load() reads it directly — no reticulate, no Python interpreter, no environment to activate.

There is exactly one Python touchpoint left, and it happens once, outside of R, before this tutorial even begins: exporting the trained model to TorchScript.

pip install ultralytics
yolo export model=yolo26s.pt format=torchscript imgsz=640

That produces a single yolo26s.torchscript file. Everything from here on — reading images, running the model, drawing boxes, summarising results — is ordinary R.

This tutorial uses YOLO26, the successor to the YOLOv10 model used in the earlier post. YOLO26 ships in n/s/m/l/x sizes (no “b”, if you’re used to YOLOv10’s naming); this post uses the small yolo26s variant. Swap in yolo26m/l/x for more accuracy at the cost of speed, using the same export command above.

Setup

install.packages(c("torch", "magick", "tidyverse", "cowplot", "exiftoolr"))
torch::install_torch() # downloads libtorch itself, once

torch::install_torch() downloads the libtorch binaries for your platform. It’s a few hundred megabytes and only needs to happen once per machine, but do it before class if you’re on a slow connection.

library(torch)
library(magick)
library(tidyverse)
library(cowplot)
library(exiftoolr)
library(here)

Data

We’ll use the same street-level images from Rio de Janeiro, Brazil as the earlier post, collected through Kartaview. See that post’s Data section for a fuller discussion of the imagery and its EXIF metadata; here we just load the file paths.

img_files <- here("tutorials_datasets", "rio_kartaview_images", "geotagged") %>%
              list.files(full.names = TRUE, pattern = "\\.jpg$")

length(img_files)
# [1] 484

Loading the model

model <- jit_load("yolo26s.torchscript") # sits alongside this .Rmarkdown file

jit_load() reads the exported TorchScript graph directly; there’s no ultralytics$YOLO() wrapper here, so we lose some conveniences it provides (like to_df() from the earlier post) and have to rebuild them ourselves in plain R. That’s most of the rest of this post.

YOLO26, like most object detectors trained on it, is trained on the COCO (Common Objects in Context) dataset, so its outputs are indexed against COCO’s 80 categories. The exported model gives us class indices, not names, so we need the lookup table ourselves — Ultralytics doesn’t ship it inside the TorchScript file the way it does with the Python ultralytics package.

coco_names <- c(
    "person","bicycle","car","motorcycle","airplane","bus","train","truck","boat",
    "traffic light","fire hydrant","stop sign","parking meter","bench","bird","cat",
    "dog","horse","sheep","cow","elephant","bear","zebra","giraffe","backpack",
    "umbrella","handbag","tie","suitcase","frisbee","skis","snowboard","sports ball",
    "kite","baseball bat","baseball glove","skateboard","surfboard","tennis racket",
    "bottle","wine glass","cup","fork","knife","spoon","bowl","banana","apple",
    "sandwich","orange","broccoli","carrot","hot dog","pizza","donut","cake","chair",
    "couch","potted plant","bed","dining table","toilet","tv","laptop","mouse",
    "remote","keyboard","cell phone","microwave","oven","toaster","sink",
    "refrigerator","book","clock","vase","scissors","teddy bear","hair drier",
    "toothbrush"
)

Preparing an image for the model

ultralytics$YOLO() in the earlier post quietly resized, padded, and normalised every image before handing it to the network. jit_load() gives us none of that — the TorchScript graph expects a specific input shape, and it’s on us to produce it.

YOLO models are trained on square inputs, so a non-square photo has to be resized to fit inside a square canvas and padded with grey on the remaining sides, rather than stretched — otherwise every detected box would come out distorted. This is called letterboxing, by analogy with the black bars added to widescreen video on an old square TV.

IMG_SIZE    <- 640L
CONF_THRESH <- 0.5
PAD_COLOR   <- "#727272"   # 114/255 grey -- Ultralytics' own letterbox pad value

letterbox <- function(path, size = IMG_SIZE) {
    img  <- image_read(path)
    info <- image_info(img)
    w <- info$width; h <- info$height
    r <- min(size / w, size / h)
    new_w <- round(w * r); new_h <- round(h * r)
    resized <- image_resize(img, paste0(new_w, "x", new_h, "!"))

    pad_w <- size - new_w; pad_h <- size - new_h
    left  <- pad_w %/% 2L; top <- pad_h %/% 2L
    canvas <- image_blank(size, size, color = PAD_COLOR)
    padded <- image_composite(canvas, resized, offset = sprintf("+%d+%d", left, top))

    list(image = padded, scale = r, left = left, top = top, orig_w = w, orig_h = h)
}

We hang on to scale, left, and top because every box the model returns will be in this padded 640×640 space — we’ll need them again to map boxes back to the original image’s pixel coordinates.

Next, the padded image has to become a tensor: a 4-dimensional array of numbers (batch, channel, height, width) that torch can feed to the network.

This step has a genuine trap in it. magick::image_data() reports its array’s dimensions as (channel, width, height), which matches how you index into it (raw[channel, x, y]) — but the underlying byte buffer that as.integer() reads out is actually laid out (height, width, channel), column-major. Reshaping with the dimensions image_data() reports, the “obvious” thing to do, silently scrambles the pixels: every box still comes back and parses without error, but confidences quietly collapse (in testing, from around 0.95 to around 0.19) with no warning or error to flag it. A square test image can’t catch this, because it can’t distinguish a width-major reshape from a height-major one — you need an asymmetric image and a pixel of known colour at a known, non-symmetric coordinate to notice the bug at all.

image_to_tensor <- function(img) {
    raw <- image_data(img, channels = "rgb")
    arr <- array(as.integer(raw), dim = rev(dim(raw)))  # true storage order: (H, W, C)
    arr <- aperm(arr, c(3, 1, 2))                        # -> (C, H, W), what torch expects
    torch_tensor(arr, dtype = torch_float32())$div(255)$unsqueeze(1)
}

Finally, since every box comes back in letterboxed 640×640 coordinates, we need the inverse of letterbox() to place them back onto the original image.

unletterbox_boxes <- function(boxes, lb) {
    boxes[, c(1, 3)] <- (boxes[, c(1, 3)] - lb$left) / lb$scale
    boxes[, c(2, 4)] <- (boxes[, c(2, 4)] - lb$top)  / lb$scale
    boxes[, 1] <- pmax(boxes[, 1], 0);        boxes[, 2] <- pmax(boxes[, 2], 0)
    boxes[, 3] <- pmin(boxes[, 3], lb$orig_w); boxes[, 4] <- pmin(boxes[, 4], lb$orig_h)
    boxes
}

Running the model on a single photograph

detect_image <- function(path, conf_thresh = CONF_THRESH) {
    lb <- letterbox(path)
    x  <- image_to_tensor(lb$image)

    out  <- with_no_grad(model(x))    # (1, 300, 6): x1, y1, x2, y2, conf, cls
    pred <- as.matrix(out$squeeze(1))

    keep <- pred[, 5] > conf_thresh
    if (!any(keep)) {
        return(tibble(file = character(), xmin = double(), ymin = double(),
                       xmax = double(), ymax = double(), conf = double(), class = character()))
    }

    boxes <- unletterbox_boxes(pred[keep, 1:4, drop = FALSE], lb)

    tibble(
        file  = path,
        xmin  = boxes[, 1], ymin = boxes[, 2],
        xmax  = boxes[, 3], ymax = boxes[, 4],
        conf  = pred[keep, 5],
        class = coco_names[pred[keep, 6] + 1]   # export is 0-indexed
    )
}

The if (!any(keep)) branch matters more than it looks. Point this at a wall, a tunnel, or a blank stretch of highway — not everything in a street-level imagery dataset has a car or a person in it — and nothing clears the confidence threshold. Returning a tibble with the right columns and zero rows (rather than a bare, columnless tibble()) means every downstream step — map_dfr(), filter(), plotting — keeps working whether or not this particular photo had anything detectable in it. It’s an easy thing to get wrong the first time and only notice once you run the function over enough real, messy images that one of them comes back empty.

Let’s try it on one photograph.

example_img <- img_files[str_detect(img_files, "1318411189")]
detections  <- detect_image(example_img)
detections %>% select(-file) %>% arrange(desc(conf))
# # A tibble: 5 x 6
#    xmin  ymin  xmax  ymax  conf class        
#   <dbl> <dbl> <dbl> <dbl> <dbl> <chr>        
# 1 1121. 1567. 1161. 1651. 0.805 traffic light
# 2 1991. 1048. 2809. 2400. 0.743 person       
# 3  769. 1999. 1057. 2180. 0.701 bus          
# 4  310. 2061.  748. 2375. 0.682 car          
# 5  535. 1998.  812. 2151. 0.649 car

Five detections, above 0.5 confidence: a person, a bus, two cars, a traffic light. This is the same shape of result to_df() handed us for free in the earlier post — we’ve just built it ourselves.

Plotting the detections

The same ggplot coordinate-system mismatch from the earlier post applies here: ggplot’s origin is bottom-left with y increasing upward, while image pixel coordinates have their origin top-left with y increasing downward. ymin/ymax have to be flipped against the image height before they land in the right place.

plot_detections <- function(path, dets,
                            classes_of_interest = c("car","person","truck","bus","motorcycle","bicycle")) {
    img  <- image_read(path)
    info <- image_info(img)
    d <- dets %>% filter(class %in% classes_of_interest)

    ggplot() +
        annotation_raster(as.raster(img), xmin = 0, xmax = info$width,
                          ymin = 0, ymax = info$height) +
        geom_rect(data = d,
                  aes(xmin = xmin, xmax = xmax,
                      ymin = info$height - ymax, ymax = info$height - ymin,
                      color = class),
                  fill = NA, linewidth = 0.8) +
        geom_text(data = d,
                  aes(x = xmin, y = info$height - ymin,
                      label = sprintf("%s %.2f", class, conf)),
                  color = "white", size = 3, hjust = 0, vjust = -0.3) +
        coord_fixed(xlim = c(0, info$width), ylim = c(0, info$height), expand = FALSE) +
        theme_void() + theme(legend.position = "none")
}

plot_detections(example_img, detections)

Batching across images

The whole point of doing this in R rather than by hand is running it across many images at once, so let’s do that — and pull in the EXIF GPS tags along the way, exactly as the earlier post did.

set.seed(42)
sample_files <- sample(img_files, 20)

detections_all <- map_dfr(sample_files, detect_image)

exif <- exif_read(sample_files, tags = c("GPSLatitude", "GPSLongitude")) %>%
    as_tibble() %>%
    select(file = SourceFile, GPSLatitude, GPSLongitude)

detections_all <- detections_all %>% left_join(exif, by = "file")

summary_counts <- detections_all %>%
    mutate(file = basename(file)) %>%
    count(file, class, name = "n")
summary_counts
# # A tibble: 28 x 3
#    file                                        class      n
#    <chr>                                       <chr>  <int>
#  1 1318425657__3645517_a809e_60c0eff7136f9.jpg car        1
#  2 1319605281__3652445_9b168_60c1dba250012.jpg bus        1
#  3 1319605281__3652445_9b168_60c1dba250012.jpg car        1
#  4 1319605281__3652445_9b168_60c1dba250012.jpg person     1
#  5 325783917__1304693_e8693_869.jpg            car        4
#  6 325858321__1304733_4ea5b_1712.jpg           car        4
#  7 325858793__1304733_4ea5b_1948.jpg           bus        1
#  8 325858793__1304733_4ea5b_1948.jpg           car        4
#  9 325961249__1304975_0a9ed_4340.jpg           car        5
# 10 325961249__1304975_0a9ed_4340.jpg           person     3
# # i 18 more rows

detections_all %>% count(class, sort = TRUE)
# # A tibble: 5 x 2
#   class             n
#   <chr>         <int>
# 1 car              58
# 2 person           11
# 3 bus               5
# 4 traffic light     2
# 5 bench             1

car dominates, as it did in the earlier post’s Rio sample — unsurprising for street-level imagery of a car-oriented corridor.

Here’s a small multiple of six images from that sample, plotted the same way as above, to get a feel for how the model does across a range of scenes.

plot_list <- map(sample_files[1:6], function(p) {
  plot_detections(p, detect_image(p))
})
plot_grid(plotlist = plot_list, nrow = 2, ncol = 3)

Note the empty panels — a tunnel, a wall glimpsed mid-motion — where nothing cleared the confidence threshold, and the doubled-up labels where two boxes overlap closely enough that their text collides. Both are exactly the kind of real-world messiness that a single cherry-picked example photo hides.

Two things worth knowing if you adapt this for your own model:

jit_load()’d TorchScript modules don’t reliably support model$eval() / model$train(FALSE) as of torch 0.17.0 + libtorch 2.8.0 — it fails with an internal S3-dispatch error on scripted modules. This particular export is safe to skip that call on: Ultralytics' export log reports the model already “fused” (BatchNorm folded into the convolution weights, zero gradient layers), and a side-by-side Python comparison calling vs. skipping model.eval() on the same file gives bit-identical output. If you export a custom or non-fused model, this is worth re-checking rather than assuming.
This exact pipeline, run against Ultralytics' own bus.jpg sample image, matched the Python/ultralytics package’s output closely: 0.97 vs 0.93 confidence on the bus, and 0.94/0.94/0.91/0.72 vs 0.93/0.94/0.44/0.43 on four people in frame. The R and Python paths aren’t doing identical floating-point work — JPEG decoding and resizing differ slightly between magick and Python’s imaging stack — so don’t expect bit-identical confidences, but the two should agree closely enough to trust the R-only path.

Exercise

Lower CONF_THRESH to 0.25 and re-run the single-image example. What shows up that wasn’t there before, and would you trust it?
Run detect_image() over the full img_files list (all of them, not just a sample of 20) and map the count of person detections per image against its GPS coordinates. Where are pedestrians most and least visible in this sample?
Compare this model’s detections on the same photograph against the YOLOv10 results from the earlier post. Do the two models agree on classes and confidences? Where do they diverge?
The classes_of_interest filter in plot_detections() silently drops everything else the model found. Remove the filter for one image — what else is in a typical street scene that a transportation-focused analysis would otherwise never see?

Conclusion

Dropping reticulate and a Python environment out of the pipeline trades one kind of complexity for another. There’s no conda environment to set up, no network drive to map, no use_condaenv() path to get right for Windows versus Mac — but in exchange, R takes on work that ultralytics$YOLO() used to do invisibly: letterboxing, tensor layout, box coordinates, even the COCO class names. Getting any one of those wrong, as the pixel-buffer reshape above shows, doesn’t necessarily throw an error — it can just quietly degrade every confidence score and leave you debugging a model, not a bug.

That tradeoff aside, the caveats from the earlier post apply just as much here, model architecture notwithstanding. This is still a pre-trained model applied with no fine-tuning, still liable to misclassify, still liable to miss things in fog, poor lighting, or partial occlusion, still trained on a dataset (COCO) whose composition shapes what it notices and what it doesn’t. Whether the inference happens in Python or in R has no bearing on any of that. Treat what comes out of either pipeline as a starting point for human review, not a finished count.

torch | Nikhil Kaza