Default page

Edit File: vmod.html

<!DOCTYPE html>

<title>VMOD - Varnish Modules — Varnish version 7.5.0 documentation</title>
 <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
 <link rel="stylesheet" type="text/css" href="../_static/classic.css?v=e2714048" />
 
 <script src="../_static/documentation_options.js?v=8214db13"></script>
 <script src="../_static/doctools.js?v=888ff710"></script>
 <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
 
 <link rel="index" title="Index" href="../genindex.html" />
 <link rel="search" title="Search" href="../search.html" />
 <link rel="next" title="VEXT - Varnish Extensions" href="vext.html" />
 <link rel="prev" title="Shell Tricks" href="shell_tricks.html" /> 
 </head><body>
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 accesskey="I">index</a></li>
 <li class="right" >
 <a href="vext.html" title="VEXT - Varnish Extensions"
 accesskey="N">next</a> |</li>
 <li class="right" >
 <a href="shell_tricks.html" title="Shell Tricks"
 accesskey="P">previous</a> |</li>
 <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 7.5.0 documentation</a> »</li>
 <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">The Varnish Reference Manual</a> »</li>
 <li class="nav-item nav-item-this"><a href="">VMOD - Varnish Modules</a></li> 
 </ul>
 </div>

<div class="document">
 <div class="documentwrapper">
 <div class="bodywrapper">
 <div class="body" role="main">
 
 <section id="vmod-varnish-modules">
<h1>VMOD - Varnish Modules<a class="headerlink" href="#vmod-varnish-modules" title="Link to this heading">¶</a></h1>
For all you can do in VCL, there are things you can not do.
Look an IP number up in a database file for instance.
VCL provides for inline C code, and there you can do everything,
but it is not a convenient or even readable way to solve such
problems.
This is where VMODs come into the picture: A VMOD is a shared
library with some C functions which can be called from VCL code.
For instance:
<div class="highlight-default notranslate"><div class="highlight"><pre>import std;

sub vcl_deliver {
 set resp.http.foo = std.toupper(req.url);
}
</pre></div>
</div>
The “std” vmod is one you get with Varnish, it will always be there
and we will put “boutique” functions in it, such as the “toupper”
function shown above. The full contents of the “std” module is
documented in vmod_std(3).
This part of the manual is about how you go about writing your own
VMOD, how the language interface between C and VCC works, where you
can find contributed VMODs etc. This explanation will use the “std”
VMOD as example, having a Varnish source tree handy may be a good
idea.
<section id="vmod-directory">
<h2>VMOD Directory<a class="headerlink" href="#vmod-directory" title="Link to this heading">¶</a></h2>
The VMOD directory is an up-to-date compilation of maintained
extensions written for Varnish Cache:
<blockquote>
<div><a class="reference external" href="https://www.varnish-cache.org/vmods">https://www.varnish-cache.org/vmods</a>
</div></blockquote>
</section>
<section id="the-vmod-vcc-file">
<h2>The vmod.vcc file<a class="headerlink" href="#the-vmod-vcc-file" title="Link to this heading">¶</a></h2>
The interface between your VMOD and the VCL compiler (“VCC”) and the
VCL runtime (“VRT”) is defined in the vmod.vcc file which a python
script called “vmodtool.py” turns into thaumaturgically challenged C
data structures that do all the hard work.
The std VMODs vmod.vcc file looks somewhat like this:
<div class="highlight-default notranslate"><div class="highlight"><pre>$ABI strict
$Module std 3 "Varnish Standard Module"
$Event event_function
$Function STRING toupper(STRANDS s)
$Function STRING tolower(STRANDS s)
$Function VOID set_ip_tos(INT)
</pre></div>
</div>
The <code class="docutils literal notranslate">$ABI</code> line is optional. Possible values are <code class="docutils literal notranslate">strict</code>
(default) and <code class="docutils literal notranslate">vrt</code>. It allows to specify that a vmod is integrating
with the blessed <code class="docutils literal notranslate">vrt</code> interface provided by <code class="docutils literal notranslate">varnishd</code> or go
deeper in the stack.
As a rule of thumb you, if the VMOD uses more than the VRT (Varnish
RunTime), in which case it needs to be built for the exact Varnish
version, use <code class="docutils literal notranslate">strict</code>. If it complies to the VRT and only needs
to be rebuilt when breaking changes are introduced to the VRT API,
use <code class="docutils literal notranslate">vrt</code>.
The <code class="docutils literal notranslate">$Module</code> line gives the name of the module, the manual section
where the documentation will reside, and the description.
The <code class="docutils literal notranslate">$Event</code> line specifies an optional “Event” function, which
will be called whenever a VCL program which imports this VMOD is
loaded or transitions to any of the warm, active, cold or discarded
states. More on this below.
The <code class="docutils literal notranslate">$Function</code> lines define three functions in the VMOD, along
with the types of the arguments, and that is probably where the
hardest bit of writing a VMOD is to be found, so we will talk about
that at length in a moment.
Notice that the third function returns VOID, that makes it a “procedure”
in VCL lingo, meaning that it cannot be used in expressions, right side
of assignments and such. Instead it can be used as a primary action,
something functions which return a value can not:
<div class="highlight-default notranslate"><div class="highlight"><pre>sub vcl_recv {
 std.set_ip_tos(32);
}
</pre></div>
</div>
Running vmodtool.py on the vmod.vcc file, produces a “vcc_if.c” and
“vcc_if.h” files, which you must use to build your shared library
file.
Forget about vcc_if.c everywhere but your Makefile, you will never
need to care about its contents, and you should certainly never
modify it, that voids your warranty instantly.
But vcc_if.h is important for you, it contains the prototypes for
the functions you want to export to VCL.
For the std VMOD, the compiled vcc_if.h file looks like this:
<div class="highlight-default notranslate"><div class="highlight"><pre>VCL_STRING vmod_toupper(VRT_CTX, VCL_STRANDS);
VCL_STRING vmod_tolower(VRT_CTX, VCL_STRANDS);
VCL_VOID vmod_set_ip_tos(VRT_CTX, VCL_INT);

vmod_event_f event_function;
</pre></div>
</div>
Those are your C prototypes. Notice the <code class="docutils literal notranslate">vmod_</code> prefix on the
function names.
<section id="named-arguments-and-default-values">
<h3>Named arguments and default values<a class="headerlink" href="#named-arguments-and-default-values" title="Link to this heading">¶</a></h3>
The basic vmod.vcc function declaration syntax introduced above makes all
arguments mandatory for calls from vcl - which implies that they need
to be given in order.
Naming the arguments as in:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Function BOOL match_acl(ACL acl, IP ip)
</pre></div>
</div>
allows calls from VCL with named arguments in any order, for example:
<div class="highlight-default notranslate"><div class="highlight"><pre>if (debug.match_acl(ip=client.ip, acl=local)) { # ...
</pre></div>
</div>
Named arguments also take default values, so for this example from
the debug vmod:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Function STRING argtest(STRING one, REAL two=2, STRING three="3",
 STRING comma=",", INT four=4)
</pre></div>
</div>
only argument <cite>one</cite> is required, so that all of the following are
valid invocations from vcl:
<div class="highlight-default notranslate"><div class="highlight"><pre>debug.argtest("1", 2.1, "3a")
debug.argtest("1", two=2.2, three="3b")
debug.argtest("1", three="3c", two=2.3)
debug.argtest("1", 2.4, three="3d")
debug.argtest("1", 2.5)
debug.argtest("1", four=6);
</pre></div>
</div>
The C interface does not change with named arguments and default
values, arguments remain positional and default values appear no
different to user specified values.
<cite>Note</cite> that default values have to be given in the native C-type
syntax, see below. As a special case, <code class="docutils literal notranslate">NULL</code> has to be given as <code class="docutils literal notranslate">0</code>.
</section>
<section id="optional-arguments">
<h3>Optional arguments<a class="headerlink" href="#optional-arguments" title="Link to this heading">¶</a></h3>
The vmod.vcc declaration also allows for optional arguments in square
brackets like so:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Function VOID opt(PRIV_TASK priv, INT four = 4, [STRING opt])
</pre></div>
</div>
With any optional argument present, the C function prototype looks
completely different:
<blockquote>
<div><ul class="simple">
<li>Only the <code class="docutils literal notranslate">VRT_CTX</code> and object pointer arguments (only for
methods) remain positional</li>
<li>All other arguments get passed in a struct as the last
argument of the C function.</li>
</ul>
</div></blockquote>
The argument struct is simple, vmod authors should check the
<cite>vmodtool</cite>-generated <code class="docutils literal notranslate">vcc_if.c</code> file for the function and struct
declarations:
<blockquote>
<div><ul>
<li>for each optional argument, a <code class="docutils literal notranslate">valid_</code><cite>argument</cite> member
is used to signal the presence of the respective optional
argument.
<code class="docutils literal notranslate">valid_</code> argstruct members should only be used as truth
values, irrespective of their actual data type.
</li>
<li>named arguments are passed in argument struct members by the
same name and with the same data type.</li>
<li>unnamed (positional) arguments are passed as <code class="docutils literal notranslate">arg</code><cite>n</cite>
with <cite>n</cite> starting at 1 and incrementing with the argument’s
position.</li>
</ul>
</div></blockquote>
</section>
<section id="objects-and-methods">
<h3>Objects and methods<a class="headerlink" href="#objects-and-methods" title="Link to this heading">¶</a></h3>
Varnish also supports a simple object model for vmods. Objects and
methods are declared in the vcc file as:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Object class(...)
$Method .method(...)
</pre></div>
</div>
For declared object classes of a vmod, object instances can then be
created in <code class="docutils literal notranslate">vcl_init { }</code> using the <code class="docutils literal notranslate">new</code> statement:
<div class="highlight-default notranslate"><div class="highlight"><pre>sub vcl_init {
 new foo = vmod.class(...);
}
</pre></div>
</div>
and have their methods called anywhere (including in <code class="docutils literal notranslate">vcl_init {}</code>
after the instantiation):
<div class="highlight-default notranslate"><div class="highlight"><pre>sub somewhere {
 foo.method(...);
}
</pre></div>
</div>
Nothing prevents a method to be named like the constructor and the
meaning of such a method is up to the vmod author:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Object foo(...)
$Method .bar(...)
$Method .foo(...)
</pre></div>
</div>
Object instances are represented as pointers to vmod-implemented C
structs. Varnish only provides space to store the address of object
instances and ensures that the right object address gets passed to C
functions implementing methods.
<blockquote>
<div><ul class="simple">
<li>Objects’ scope and lifetime are the vcl</li>
<li>Objects can only be created in <code class="docutils literal notranslate">vcl_init {}</code> and have
their destructors called by varnish after <code class="docutils literal notranslate">vcl_fini {}</code>
has completed.</li>
</ul>
</div></blockquote>
vmod authors are advised to understand the prototypes in the
<cite>vmodtool</cite>-generated <code class="docutils literal notranslate">vcc_if.c</code> file:
<blockquote>
<div><ul class="simple">
<li>For <code class="docutils literal notranslate">$Object</code> declarations, a constructor and destructor
function must be implemented</li>
<li>The constructor is named by the suffix <code class="docutils literal notranslate">__init</code>, always is
of <code class="docutils literal notranslate">VOID</code> return type and has the following arguments
before the vcc-declared parameters:
<ul>
<li><code class="docutils literal notranslate">VRT_CTX</code> as usual</li>
<li>a pointer-pointer to return the address of the created
oject</li>
<li>a string containing the vcl name of the object instance</li>
</ul>
</li>
<li>The destructor is named by the suffix <code class="docutils literal notranslate">__fini</code>, always is
of <code class="docutils literal notranslate">VOID</code> return type and has a single argument, the
pointer-pointer to the address of the object. The destructor
is expected clear the address of the object stored in that
pointer-pointer.</li>
<li><dl class="simple">
<dt>Methods gain the pointer to the object as an argument after</dt><dd>the <code class="docutils literal notranslate">VRT_CTX</code>.
</dd>
</dl>
</li>
</ul>
</div></blockquote>
As varnish is in no way involved in managing object instances other
than passing their addresses, vmods need to implement all aspects of
managing instances, in particular their memory management. As the
lifetime of object instances is the vcl, they will usually be
allocated from the heap.
</section>
<section id="functions-and-methods-scope-restriction">
<h3>Functions and Methods scope restriction<a class="headerlink" href="#functions-and-methods-scope-restriction" title="Link to this heading">¶</a></h3>
The <code class="docutils literal notranslate">$Restrict</code> stanza offers a way to limit the scope of the preceding vmod function
or method, so that they can only be called from restricted vcl call sites.
It must only appear after a <code class="docutils literal notranslate">$Method</code> or <code class="docutils literal notranslate">$Function</code> and has the following syntax:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Restrict scope1 [scope2 ...]
</pre></div>
</div>
Possible scope values are:
<code class="docutils literal notranslate">backend, client, housekeeping, vcl_recv, vcl_pipe, vcl_pass, vcl_hash, vcl_purge, vcl_miss, vcl_hit,
vcl_deliver, vcl_synth, vcl_backend_fetch, vcl_backend_response, vcl_backend_error, vcl_init, vcl_fini</code>
</section>
<section id="deprecated-aliases">
<h3>Deprecated Aliases<a class="headerlink" href="#deprecated-aliases" title="Link to this heading">¶</a></h3>
The <code class="docutils literal notranslate">$Alias</code> stanza offers a mechanism to rename a function or an
object’s method without removing the previous name. This allows name
changes to maintain compatibility until the alias is dropped.
The syntax for a function is:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Alias deprecated_function original_function

[description]
</pre></div>
</div>
The syntax for a method is:
<div class="highlight-default notranslate"><div class="highlight"><pre>$Alias .deprecated_method object.original_method

[description]
</pre></div>
</div>
The <code class="docutils literal notranslate">$Alias</code> stanza can appear anywhere, this allows grouping them
in a dedicated “deprecated” section of their manual. The optional
description can be used to explain why a function was renamed.
</section>
</section>
<section id="vcl-and-c-data-types">
<h2>VCL and C data types<a class="headerlink" href="#vcl-and-c-data-types" title="Link to this heading">¶</a></h2>
VCL data types are targeted at the job, so for instance, we have data
types like “DURATION” and “HEADER”, but they all have some kind of C
language representation. Here is a description of them.
All but the PRIV types have typedefs: VCL_INT, VCL_REAL, etc.
Notice that most of the non-native (C pointer) types are <code class="docutils literal notranslate">const</code>,
which, if returned by a vmod function/method, are assumed to be
immutable. In other words, a vmod <cite>must not</cite> modify any data which was
previously returned.
When returning non-native values, the producing function is
responsible for arranging memory management. Either by freeing the
structure later by whatever means available or by using storage
allocated from the client or backend workspaces.
<dl>
<dt>ACL</dt><dd>C-type: <code class="docutils literal notranslate">const struct vrt_acl *</code>
A type for named ACLs declared in VCL.
</dd>
<dt>BACKEND</dt><dd>C-type: <code class="docutils literal notranslate">const struct director *</code>
A type for backend and director implementations. See
<a class="reference internal" href="directors.html#ref-writing-a-director">Writing a Director</a>.
</dd>
<dt>BLOB</dt><dd>C-type: <code class="docutils literal notranslate">const struct vmod_priv *</code>
An opaque type to pass random bits of memory between VMOD
functions.
</dd>
<dt>BODY</dt><dd>C-type: <code class="docutils literal notranslate">const void *</code>
A type only used on the LHS of an assignment that can take
either a blob or an expression that can be converted to a
string.
</dd>
<dt>BOOL</dt><dd>C-type: <code class="docutils literal notranslate">unsigned</code>
Zero means false, anything else means true.
</dd>
<dt>BYTES</dt><dd>C-type: <code class="docutils literal notranslate">double</code>
Unit: bytes.
A storage space, as in 1024 bytes.
</dd>
<dt>DURATION</dt><dd>C-type: <code class="docutils literal notranslate">double</code>
Unit: seconds.
A time interval, as in 25 seconds.
</dd>
<dt>ENUM</dt><dd>vcc syntax: ENUM { val1, val2, … }
vcc example: <code class="docutils literal notranslate">ENUM { one, two, three } number="one"</code>
C-type: <code class="docutils literal notranslate">const char *</code>
Allows values from a set of constant strings. <cite>Note</cite> that the
C-type is a string, not a C enum.
Enums will be passed as fixed pointers, so instead of string
comparisons, also pointer comparisons with <code class="docutils literal notranslate">VENUM(name)</code> are
possible.
</dd>
<dt>HEADER</dt><dd>C-type: <code class="docutils literal notranslate">const struct gethdr_s *</code>
These are VCL compiler generated constants referencing a
particular header in a particular HTTP entity, for instance
<code class="docutils literal notranslate">req.http.cookie</code> or <code class="docutils literal notranslate">beresp.http.last-modified</code>. By passing
a reference to the header, the VMOD code can both read and write
the header in question.
If the header was passed as STRING, the VMOD code only sees
the value, but not where it came from.
</dd>
<dt>HTTP</dt><dd>C-type: <code class="docutils literal notranslate">struct http *</code>
A reference to a header object as <code class="docutils literal notranslate">req.http</code> or <code class="docutils literal notranslate">bereq.http</code>.
</dd>
<dt>INT</dt><dd>C-type: <code class="docutils literal notranslate">long</code>
A (long) integer as we know and love them.
</dd>
<dt>IP</dt><dd>C-type: <code class="docutils literal notranslate">const struct suckaddr *</code>
This is an opaque type, see the <code class="docutils literal notranslate">include/vsa.h</code> file for
which primitives we support on this type.
</dd>
<dt>PRIV_CALL</dt><dd>See <a class="reference internal" href="#ref-vmod-private-pointers">Private Pointers</a> below.
</dd>
<dt>PRIV_TASK</dt><dd>See <a class="reference internal" href="#ref-vmod-private-pointers">Private Pointers</a> below.
</dd>
<dt>PRIV_TOP</dt><dd>See <a class="reference internal" href="#ref-vmod-private-pointers">Private Pointers</a> below.
</dd>
<dt>PRIV_VCL</dt><dd>See <a class="reference internal" href="#ref-vmod-private-pointers">Private Pointers</a> below.
</dd>
<dt>PROBE</dt><dd>C-type: <code class="docutils literal notranslate">const struct vrt_backend_probe *</code>
A named standalone backend probe definition.
</dd>
<dt>REAL</dt><dd>C-type: <code class="docutils literal notranslate">double</code>
A floating point value.
</dd>
<dt>REGEX</dt><dd>C-type: <code class="docutils literal notranslate">const struct vre *</code>
This is an opaque type for regular expressions with a VCL scope.
The REGEX type is only meant for regular expression literals
managed by the VCL compiler. For dynamic regular expressions or
complex usage see the API from the <code class="docutils literal notranslate">include/vre.h</code> file.
</dd>
<dt>STRING</dt><dd>C-type: <code class="docutils literal notranslate">const char *</code>
A NUL-terminated text-string.
Can be NULL to indicate a nonexistent string, for instance in:
<div class="highlight-default notranslate"><div class="highlight"><pre>mymod.foo(req.http.foobar);
</pre></div>
</div>
If there were no “foobar” HTTP header, the vmod_foo()
function would be passed a NULL pointer as argument.
</dd>
<dt>STEVEDORE</dt><dd>C-type: <code class="docutils literal notranslate">const struct stevedore *</code>
A storage backend.
</dd>
<dt>STRANDS</dt><dd>C-Type: <code class="docutils literal notranslate">const struct strands *</code>
Strands are a list of strings that gets passed in a struct with the
following members:
<ul class="simple">
<li><code class="docutils literal notranslate">int n</code>: the number of strings</li>
<li><code class="docutils literal notranslate">const char **p</code>: the array of strings with <cite>n</cite> elements</li>
</ul>
A VMOD should never hold onto strands beyond a function or method
execution. See <code class="docutils literal notranslate">include/vrt.h</code> for the details.
</dd>
<dt>TIME</dt><dd>C-type: <code class="docutils literal notranslate">double</code>
Unit: seconds since UNIX epoch.
An absolute time, as in 1284401161.
</dd>
<dt>VCL_SUB</dt><dd>C-type: <code class="docutils literal notranslate">const struct vcl_sub *</code>
Opaque handle on a VCL subroutine.
References to subroutines can be passed into VMODs as
arguments and called later through <code class="docutils literal notranslate">VRT_call()</code>. The scope
strictly is the VCL: vmods must ensure that <code class="docutils literal notranslate">VCL_SUB</code>
references never be called from a different VCL.
<code class="docutils literal notranslate">VRT_call()</code> fails the VCL for recursive calls and when the
<code class="docutils literal notranslate">VCL_SUB</code> can not be called from the current context
(e.g. calling a subroutine accessing <code class="docutils literal notranslate">req</code> from the backend
side).
For more than one invocation of <code class="docutils literal notranslate">VRT_call()</code>, VMODs must
check if <code class="docutils literal notranslate">VRT_handled()</code> returns non-zero inbetween calls:
The called SUB may have returned with an action (any
<code class="docutils literal notranslate">return(x)</code> other than plain <code class="docutils literal notranslate">return</code>) or may have failed
the VCL, and in both cases the calling VMOD must return
also, possibly after having conducted some cleanup. Note that
undoing the handling through <code class="docutils literal notranslate">VRT_handling()</code> is a bug.
<code class="docutils literal notranslate">VRT_check_call()</code> can be used to check if a <code class="docutils literal notranslate">VRT_call()</code>
would succeed in order to avoid the potential VCL failure. It
returns <code class="docutils literal notranslate">NULL</code> if <code class="docutils literal notranslate">VRT_call()</code> would make the call or an
error string why not.
</dd>
<dt>VOID</dt><dd>C-type: <code class="docutils literal notranslate">void</code>
Can only be used for return-value, which makes the function a VCL
procedure.
</dd>
</dl>
</section>
<section id="private-pointers">
<h2>Private Pointers<a class="headerlink" href="#private-pointers" title="Link to this heading">¶</a></h2>
It is often useful for library functions to maintain local state,
this can be anything from a precompiled regexp to open file descriptors
and vast data structures.
The VCL compiler supports the following private pointers:
<ul>
<li><code class="docutils literal notranslate">PRIV_CALL</code> “per call” private pointers are useful to cache/store
state relative to the specific call or its arguments, for instance a
compiled regular expression specific to a regsub() statement or
simply caching the most recent output of some expensive operation.
These private pointers live for the duration of the loaded VCL.</li>
<li><code class="docutils literal notranslate">PRIV_TASK</code> “per task” private pointers are useful for state that
applies to calls for either a specific request or a backend
request. For instance this can be the result of a parsed cookie
specific to a client. Note that <code class="docutils literal notranslate">PRIV_TASK</code> contexts are separate
for the client side and the backend side, so use in
<code class="docutils literal notranslate">vcl_backend_*</code> will yield a different private pointer from the
one used on the client side.
These private pointers live only for the duration of their task.</li>
<li><code class="docutils literal notranslate">PRIV_TOP</code> “per top-request” private pointers live for the
duration of one request and all its ESI-includes. They are only
defined for the client side. When used from backend VCL subs, a NULL
pointer will potentially be passed and a VCL failure triggered.
These private pointers live only for the duration of their top
level request
</li>
<li><code class="docutils literal notranslate">PRIV_VCL</code> “per vcl” private pointers are useful for such global
state that applies to all calls in this VCL, for instance flags that
determine if regular expressions are case-sensitive in this vmod or
similar. The <code class="docutils literal notranslate">PRIV_VCL</code> object is the same object that is passed
to the VMOD’s event function.
This private pointer lives for the duration of the loaded VCL.
The <code class="docutils literal notranslate">PRIV_CALL</code> vmod_privs are finalized before <code class="docutils literal notranslate">PRIV_VCL</code>.
</li>
</ul>
The way it works in the vmod code, is that a <code class="docutils literal notranslate">struct vmod_priv *</code> is
passed to the functions where one of the <code class="docutils literal notranslate">PRIV_*</code> argument types is
specified.
This structure contains three members:
<div class="highlight-default notranslate"><div class="highlight"><pre>struct vmod_priv {
 void *priv;
 long len;
 const struct vmod_priv_methods *methods;
};
</pre></div>
</div>
The <code class="docutils literal notranslate">.priv</code> and <code class="docutils literal notranslate">.len</code> elements can be used for whatever the vmod
code wants to use them for.
<code class="docutils literal notranslate">.methods</code> can be an optional pointer to a struct of callbacks:
<div class="highlight-default notranslate"><div class="highlight"><pre>typedef void vmod_priv_fini_f(VRT_CTX, void *);

struct vmod_priv_methods {
 unsigned magic;
 const char *type;
 vmod_priv_fini_f *fini;
};
</pre></div>
</div>
<code class="docutils literal notranslate">.magic</code> has to be initialized to
<code class="docutils literal notranslate">VMOD_PRIV_METHODS_MAGIC</code>. <code class="docutils literal notranslate">.type</code> should be a descriptive name to
help debugging.
<code class="docutils literal notranslate">.fini</code> will be called for a non-NULL <code class="docutils literal notranslate">.priv</code> of the <code class="docutils literal notranslate">struct
vmod_priv</code> when the scope ends with that <code class="docutils literal notranslate">.priv</code> pointer as its
second argument besides a <code class="docutils literal notranslate">VRT_CTX</code>.
The common case where a private data structure is allocated with
malloc(3) would look like this:
<div class="highlight-default notranslate"><div class="highlight"><pre>static void
myfree(VRT_CTX, void *p)
{
 CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 free (p);
}

static const struct vmod_priv_methods mymethods[1] = {{
 .magic = VMOD_PRIV_METHODS_MAGIC,
 .type = "mystate",
 .fini = myfree
}};

// ....

if (priv->priv == NULL) {
 priv->priv = calloc(1, sizeof(struct myfoo));
 AN(priv->priv);
 priv->methods = mymethods;
 mystate = priv->priv;
 mystate->foo = 21;
 ...
} else {
 mystate = priv->priv;
}
if (foo > 25) {
 ...
}
</pre></div>
</div>
<section id="private-pointers-memory-management">
<h3>Private Pointers Memory Management<a class="headerlink" href="#private-pointers-memory-management" title="Link to this heading">¶</a></h3>
The generic malloc(3) / free(3) approach documented above works for
all private pointers. It is the simplest and less error prone (as long
as allocated memory is properly freed though the fini callback), but
comes at the cost of calling into the heap memory allocator.
Per-vmod constant data structures can be assigned to any private
pointer type, but, obviously, free(3) must not be used on them.
Dynamic data stored in <code class="docutils literal notranslate">PRIV_TASK</code> and <code class="docutils literal notranslate">PRIV_TOP</code> pointers can
also come from the workspace:
<ul>
<li>For <code class="docutils literal notranslate">PRIV_TASK</code>, any allocation from <code class="docutils literal notranslate">ctx->ws</code> works, like so:
<div class="highlight-default notranslate"><div class="highlight"><pre>if (priv->priv == NULL) {
 priv->priv = WS_Alloc(ctx->ws, sizeof(struct myfoo));
 if (priv->priv == NULL) {
 VRT_fail(ctx, "WS_Alloc failed");
 return (...);
 }
 priv->methods = mymethods;
 mystate = priv->priv;
 mystate->foo = 21;
 ...
</pre></div>
</div>
</li>
<li>For <code class="docutils literal notranslate">PRIV_TOP</code>, first of all keep in mind that it must only be
used from the client context, so vmod code should error out for
<code class="docutils literal notranslate">ctx->req == NULL</code>.
For dynamic data, the top request’s workspace must be used, which
complicates things a bit:
<div class="highlight-default notranslate"><div class="highlight"><pre>if (priv->priv == NULL) {
 struct ws *ws;

CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 CHECK_OBJ_NOTNULL(ctx->req->top, REQTOP_MAGIC);
 CHECK_OBJ_NOTNULL(ctx->req->top->topreq, REQ_MAGIC);
 ws = ctx->req->top->topreq->ws;

priv->priv = WS_Alloc(ws, sizeof(struct myfoo));
 // ... same as above for PRIV_TASK
</pre></div>
</div>
</li>
</ul>
Notice that allocations on the workspace do not need to be freed,
their lifetime is the respective task.
</section>
<section id="private-pointers-and-objects">
<h3>Private Pointers and Objects<a class="headerlink" href="#private-pointers-and-objects" title="Link to this heading">¶</a></h3>
<code class="docutils literal notranslate">PRIV_TASK</code> and <code class="docutils literal notranslate">PRIV_TOP</code> arguments to methods are not per object
instance, but per vmod as for ordinary vmod functions. Thus, vmods
requiring per-task / per top-request state for object instances need
to implement other means to associate storage with object instances.
This is what <code class="docutils literal notranslate">VRT_priv_task()</code> / <code class="docutils literal notranslate">VRT_priv_task_get()</code> and
<code class="docutils literal notranslate">VRT_priv_top()</code> / <code class="docutils literal notranslate">VRT_priv_top_get()</code> are for:
The non-get functions either return an existing <code class="docutils literal notranslate">PRIV_TASK</code> /
<code class="docutils literal notranslate">PRIV_TOP</code> for a given <code class="docutils literal notranslate">void *</code> argument or create one. They
return <code class="docutils literal notranslate">NULL</code> in case of an allocation failure.
The <code class="docutils literal notranslate">_get()</code> functions do not create a <code class="docutils literal notranslate">PRIV_*</code>, but return either
an existing one or <code class="docutils literal notranslate">NULL</code>.
By convention, private pointers for object instance are created on the
address of the object, as in this example for a <code class="docutils literal notranslate">PRIV_TASK</code>:
<div class="highlight-default notranslate"><div class="highlight"><pre>VCL_VOID
myvmod_obj_method(VRT_CTX, struct myvmod_obj *o)
{
 struct vmod_priv *p;

p = VRT_priv_task(ctx, o);

// ... see above
</pre></div>
</div>
The <code class="docutils literal notranslate">PRIV_TOP</code> case looks identical except for calling
<code class="docutils literal notranslate">VRT_priv_top(ctx, o)</code> in place of <code class="docutils literal notranslate">VRT_priv_task(ctx, o)</code>, but be
reminded that the <code class="docutils literal notranslate">VRT_priv_top*()</code> functions must only be called
from client context (if <code class="docutils literal notranslate">ctx->req != NULL</code>).
</section>
</section>
<section id="event-functions">
<h2>Event functions<a class="headerlink" href="#event-functions" title="Link to this heading">¶</a></h2>
VMODs can have an “event” function which is called when a VCL which
imports the VMOD is loaded or discarded. This corresponds to the
<code class="docutils literal notranslate">VCL_EVENT_LOAD</code> and <code class="docutils literal notranslate">VCL_EVENT_DISCARD</code> events, respectively.
In addition, this function will be called when the VCL temperature is
changed to cold or warm, corresponding to the <code class="docutils literal notranslate">VCL_EVENT_COLD</code> and
<code class="docutils literal notranslate">VCL_EVENT_WARM</code> events.
The first argument to the event function is a VRT context.
The second argument is the vmod_priv specific to this particular VCL,
and if necessary, a VCL specific VMOD “fini” function can be attached
to its “free” hook.
The third argument is the event.
If the VMOD has private global state, which includes any sockets or files
opened, any memory allocated to global or private variables in the C-code etc,
it is the VMODs own responsibility to track how many VCLs were loaded or
discarded and free this global state when the count reaches zero.
VMOD writers are strongly encouraged to release all per-VCL resources for a
given VCL when it emits a <code class="docutils literal notranslate">VCL_EVENT_COLD</code> event. You will get a chance to
reacquire the resources before the VCL becomes active again and be notified
first with a <code class="docutils literal notranslate">VCL_EVENT_WARM</code> event. Unless a user decides that a given VCL
should always be warm, an inactive VMOD will eventually become cold and should
manage resources accordingly.
An event function must return zero upon success. It is only possible to fail
an initialization with the <code class="docutils literal notranslate">VCL_EVENT_LOAD</code> or <code class="docutils literal notranslate">VCL_EVENT_WARM</code> events.
Should such a failure happen, a <code class="docutils literal notranslate">VCL_EVENT_DISCARD</code> or <code class="docutils literal notranslate">VCL_EVENT_COLD</code>
event will be sent to the VMODs that succeeded to put them back in a cold
state. The VMOD that failed will not receive this event, and therefore must
not be left half-initialized should a failure occur.
If your VMOD is running an asynchronous background job you can hold a reference
to the VCL to prevent it from going cold too soon and get the same guarantees
as backends with ongoing requests for instance. For that, you must acquire the
reference by calling <code class="docutils literal notranslate">VRT_VCL_Prevent_Discard</code> when you receive a <code class="docutils literal notranslate">VCL_EVENT_WARM</code> and
later calling <code class="docutils literal notranslate">VRT_VCL_Allow_Discard</code> once the background job is over. Receiving a
<code class="docutils literal notranslate">VCL_EVENT_COLD</code> is your cue to terminate any background job bound to a VCL.
You can find an example of VCL references in vmod-debug:
<div class="highlight-default notranslate"><div class="highlight"><pre>priv_vcl->vclref = VRT_VCL_Prevent_Discard(ctx, "vmod-debug");
...
VRT_VCL_Allow_Discard(&ctx, &priv_vcl->vclref);
</pre></div>
</div>
In this simplified version, you can see that you need at least a VCL-bound data
structure like a <code class="docutils literal notranslate">PRIV_VCL</code> or a VMOD object to keep track of the reference
and later release it. You also have to provide a description, it will be printed
to the user if they try to warm up a cooling VCL:
<div class="highlight-default notranslate"><div class="highlight"><pre>$ varnishadm vcl.list
available auto/cooling 0 vcl1
active auto/warm 0 vcl2

$ varnishadm vcl.state vcl1 warm
Command failed with error code 300
Failed <vcl.state vcl1 auto>
Message:
 VCL vcl1 is waiting for:
 - vmod-debug
</pre></div>
</div>
In the case where properly releasing resources may take some time, you can
opt for an asynchronous worker, either by spawning a thread and tracking it, or
by using Varnish’s worker pools.
</section>
<section id="when-to-lock-and-when-not-to-lock">
<h2>When to lock, and when not to lock<a class="headerlink" href="#when-to-lock-and-when-not-to-lock" title="Link to this heading">¶</a></h2>
Varnish is heavily multithreaded, so by default VMODs must implement
their own locking to protect shared resources.
When a VCL is loaded or unloaded, the event and priv->free are
run sequentially all in a single thread, and there is guaranteed
to be no other activity related to this particular VCL, nor are
there init/fini activity in any other VCL or VMOD at this time.
That means that the VMOD init, and any object init/fini functions
are already serialized in sensible order, and won’t need any locking,
unless they access VMOD specific global state, shared with other VCLs.
Traffic in other VCLs which also import this VMOD, will be happening
while housekeeping is going on.
</section>
<section id="statistics-counters">
<h2>Statistics Counters<a class="headerlink" href="#statistics-counters" title="Link to this heading">¶</a></h2>
Starting in Varnish 6.0, VMODs can define their own counters that appear
in varnishstat.
If you’re using autotools, see the <code class="docutils literal notranslate">VARNISH_COUNTERS</code> macro in
varnish.m4 for documentation on getting your build set up.
Counters are defined in a .vsc file. The <code class="docutils literal notranslate">VARNISH_COUNTERS</code> macro
calls vsctool.py to turn a foo.vsc file into VSC_foo.c and
VSC_foo.h files, just like vmodtool.py turns foo.vcc into
vcc_foo_if.c and vcc_foo_if.h files. Similarly to the VCC files, the
generated VSC files give you a structure and functions that you can use
in your VMOD’s code to create and destroy the counters your defined. The
vsctool.py tool also generates a VSC_foo.rst file that you can
include in your documentation to describe the counters your VMOD has.
The .vsc file looks like this:
<div class="highlight-none notranslate"><div class="highlight"><pre>.. varnish_vsc_begin:: xkey
 :oneliner: xkey Counters
 :order: 70

Metrics from vmod_xkey

.. varnish_vsc:: g_keys
        :type:          gauge
        :oneliner:      Number of surrogate keys

Number of surrogate keys in use. Increases after a request that includes a new key in the xkey header. Decreases when a key is purged or when all cache objects associated with a key expire.

.. varnish_vsc_end:: xkey
</pre></div>
</div>
Counters can have the following parameters:
<dl class="simple">
<dt>type</dt><dd>The type of metric this is. Can be one of <code class="docutils literal notranslate">counter</code>,
<code class="docutils literal notranslate">gauge</code>, or <code class="docutils literal notranslate">bitmap</code>.
</dd>
<dt>ctype</dt><dd>The type that this counter will have in the C code. This can
only be <code class="docutils literal notranslate">uint64_t</code> and does not need to be specified.
</dd>
<dt>level</dt><dd>The verbosity level of this counter. varnishstat will only
show counters with a higher verbosity level than the one
currently configured. Can be one of <code class="docutils literal notranslate">info</code>, <code class="docutils literal notranslate">diag</code>, or
<code class="docutils literal notranslate">debug</code>.
</dd>
<dt>oneliner</dt><dd>A short, one line description of the counter.
</dd>
<dt>group</dt><dd>I don’t know what this does.
</dd>
<dt>format</dt><dd>Can be one of <code class="docutils literal notranslate">integer</code>, <code class="docutils literal notranslate">bytes</code>, <code class="docutils literal notranslate">bitmap</code>, or <code class="docutils literal notranslate">duration</code>.
</dd>
</dl>
After these parameters, a counter can have a longer description, though
this description has to be all on one line in the .vsc file.
You should call <code class="docutils literal notranslate">VSC_*_New()</code> when your VMOD is loaded and
<code class="docutils literal notranslate">VSC_*_Destroy()</code> when it is unloaded. See the generated
<code class="docutils literal notranslate">VSC_*.h</code> file for the full details about the structure that contains
your counters.
</section>
</section>

<div class="clearer"></div>
 </div>
 </div>
 </div>
 <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
 <div class="sphinxsidebarwrapper">
 <div>
 <h3><a href="../index.html">Table of Contents</a></h3>
 <ul>
<li><a class="reference internal" href="#">VMOD - Varnish Modules</a><ul>
<li><a class="reference internal" href="#vmod-directory">VMOD Directory</a></li>
<li><a class="reference internal" href="#the-vmod-vcc-file">The vmod.vcc file</a><ul>
<li><a class="reference internal" href="#named-arguments-and-default-values">Named arguments and default values</a></li>
<li><a class="reference internal" href="#optional-arguments">Optional arguments</a></li>
<li><a class="reference internal" href="#objects-and-methods">Objects and methods</a></li>
<li><a class="reference internal" href="#functions-and-methods-scope-restriction">Functions and Methods scope restriction</a></li>
<li><a class="reference internal" href="#deprecated-aliases">Deprecated Aliases</a></li>
</ul>
</li>
<li><a class="reference internal" href="#vcl-and-c-data-types">VCL and C data types</a></li>
<li><a class="reference internal" href="#private-pointers">Private Pointers</a><ul>
<li><a class="reference internal" href="#private-pointers-memory-management">Private Pointers Memory Management</a></li>
<li><a class="reference internal" href="#private-pointers-and-objects">Private Pointers and Objects</a></li>
</ul>
</li>
<li><a class="reference internal" href="#event-functions">Event functions</a></li>
<li><a class="reference internal" href="#when-to-lock-and-when-not-to-lock">When to lock, and when not to lock</a></li>
<li><a class="reference internal" href="#statistics-counters">Statistics Counters</a></li>
</ul>
</li>
</ul>

</div>
 <div>
 <h4>Previous topic</h4>
 <a href="shell_tricks.html"
 title="previous chapter">Shell Tricks</a>
 </div>
 <div>
 <h4>Next topic</h4>
 <a href="vext.html"
 title="next chapter">VEXT - Varnish Extensions</a>
 </div>
 <div role="note" aria-label="source link">
 <h3>This Page</h3>
 <ul class="this-page-menu">
 <li><a href="../_sources/reference/vmod.rst.txt"
 rel="nofollow">Show Source</a></li>
 </ul>
 </div>
<div id="searchbox" style="display: none" role="search">
 <h3 id="searchlabel">Quick search</h3>
 <div class="searchformwrapper">
 <form class="search" action="../search.html" method="get">
 <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
 <input type="submit" value="Go" />
 </form>
 </div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
 </div>
 </div>
 <div class="clearer"></div>
 </div>
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 >index</a></li>
 <li class="right" >
 <a href="vext.html" title="VEXT - Varnish Extensions"
 >next</a> |</li>
 <li class="right" >
 <a href="shell_tricks.html" title="Shell Tricks"
 >previous</a> |</li>
 <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 7.5.0 documentation</a> »</li>
 <li class="nav-item nav-item-1"><a href="index.html" >The Varnish Reference Manual</a> »</li>
 <li class="nav-item nav-item-this"><a href="">VMOD - Varnish Modules</a></li> 
 </ul>
 </div>
 <div class="footer" role="contentinfo">
 © Copyright 2010-2014, Varnish Software AS.
 Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 7.2.6.
 </div>
 </body>
</html>