Home Artificial Intelligence Posit AI Weblog: safetensors 0.1.0

Posit AI Weblog: safetensors 0.1.0

Posit AI Weblog: safetensors 0.1.0


safetensors is a brand new, easy, quick, and secure file format for storing tensors. The design of the file format and its unique implementation are being led
by Hugging Face, and it’s getting largely adopted of their well-liked ‘transformers’ framework. The safetensors R package deal is a pure-R implementation, permitting to each learn and write safetensor information.

The preliminary model (0.1.0) of safetensors is now on CRAN.


The principle motivation for safetensors within the Python group is safety. As famous
within the official documentation:

The principle rationale for this crate is to take away the necessity to use pickle on PyTorch which is utilized by default.

Pickle is taken into account an unsafe format, because the motion of loading a Pickle file can
set off the execution of arbitrary code. This has by no means been a priority for torch
for R customers, for the reason that Pickle parser that’s included in LibTorch solely helps a subset
of the Pickle format, which doesn’t embody executing code.

Nonetheless, the file format has further benefits over different generally used codecs, together with:

  • Assist for lazy loading: You may select to learn a subset of the tensors saved within the file.

  • Zero copy: Studying the file doesn’t require extra reminiscence than the file itself.
    (Technically the present R implementation does makes a single copy, however that may
    be optimized out if we actually want it sooner or later).

  • Easy: Implementing the file format is easy, and doesn’t require complicated dependencies.
    Because of this it’s a very good format for exchanging tensors between ML frameworks and
    between completely different programming languages. As an example, you possibly can write a safetensors file
    in R and cargo it in Python, and vice-versa.

There are further benefits in comparison with different file codecs widespread on this area, and
you possibly can see a comparability desk right here.


The safetensors format is described within the determine beneath. It’s mainly a header file
containing some metadata, adopted by uncooked tensor buffers.

Diagram describing the safetensors file format.

Primary utilization

safetensors will be put in from CRAN utilizing:

Nick Fewings on Unsplash


Textual content and figures are licensed beneath Inventive Commons Attribution CC BY 4.0. The figures which have been reused from different sources do not fall beneath this license and will be acknowledged by a word of their caption: “Determine from …”.


For attribution, please cite this work as

Falbel (2023, June 15). Posit AI Weblog: safetensors 0.1.0. Retrieved from https://blogs.rstudio.com/tensorflow/posts/2023-06-15-safetensors/

BibTeX quotation

  creator = {Falbel, Daniel},
  title = {Posit AI Weblog: safetensors 0.1.0},
  url = {https://blogs.rstudio.com/tensorflow/posts/2023-06-15-safetensors/},
  yr = {2023}



Please enter your comment!
Please enter your name here