crandas.base

This module deals with the connection to the VDL server. To set up a connection, it is recommended to manually invoke the connect() method before performing any operations in crandas. This function returns a new Session object, that stores parameters about the connection.

>>> import crandas as cd
>>> session = cd.connect("default") # This looks for `default.vdlconn`
>>> cd.DataFrame({})

Depending on the connection file, the connection is made in design mode or in authorized mode. The used mode can be inspected by printing it:

>>> session.mode

There is one global active Session object, called session. Whenever connect() is called, this variable is updated to reflect the new session object. Any operation, such as creating or opening a DataFrame, happens in the global session, except when a session is specified as a keyword argument. To temporarily set the default session to my_session, use my_session as a context handler, e.g.,

with my_session:
    cd.DataFrame() # performed in my_session

When the user does not call connect() manually, Session.connect() is lazily invoked on the existing Session object as soon as the user issues their first crandas command that needs a connection to the VDL server, like uploading a table. Because of this, the user can also set configuration parameters by modifying the existing global Session object, prior to issuing any crandas commands:

>>> import crandas as cd
>>> from crandas.base import session
>>> from pathlib import Path
>>> session.endpoint = "https://my-vdl-cluster:8920/api/v1"
>>> session.base_path = Path("/my/vdl/path")
>>> table = cd.DataFrame({})

Next to the choice between actively calling connect() with parameters, or changing the existing Session object variables, the user also has a choice between using the VDL connection file (recommended), and specifying the endpoint and base_path (legacy method). See Session for details.

exception crandas.base.DesignModeWarning

Bases: UserWarning

Warning that the server is running in design mode. Can be disabled via warn_design setting

class crandas.base.Session(base_path=None, insecure_one_party_mode=False, mode=None, analyst_key=None, keepalive=None, endpoint=None, api_token=None, override_version_check=False, **kwargs)

Bases: object

This object stores the state specific to the connection to the VDL server. To configure the connection, it is recommended to use the connect() function, that initializes a fresh instance of this class.

There are two ways to configure the connection: using a VDL connection file (recommended, new default), or by manually specifying an URL endpoint and a base folder that contains all required key material (legacy approach).

If the user does not specify a VDL connection file nor a base path explicitly, the default behavior is to look for the default.vdlconn file (overridable through the default_connection_file variable, see crandas.config), inside of the configuration folder (see crandas.config). If it is not present, the configuration folder is scanned for files with the .vdlconn extension. If there is a single connection file, that file is used. If there are multiple connection files, the system does not know which file to choose and it raises a crandas.errors.ClientError.

Besides the VDL connection file, the system needs an analyst key when connecting to a cluster that is in authorized mode. By default, the analyst key is taken from analyst.sk from the configuration folder (unless legacy mode is used, then it is clientsign.sk for backwards compatibility). It can be configured using the analyst_key argument:

>>> import crandas as cd
>>> from pathlib import Path
>>> session = cd.connect("default", analyst_key=Path("/path/to/my/analyst.sk"))
>>> crandas.DataFrame({})

The user can also specify the server endpoint and key material manually (legacy approach). When the user changes the endpoint attribute of an existing Session object, that Session object is set to legacy mode. In legacy mode, the key material is read from files that are specified relative to the base_path attribute. Note that if the connect() method is not actively called by the user, it is lazily called on the first crandas method that requires a connection to the VDL. This provides a way to first configure the variables in legacy mode; for example:

>>> import crandas as cd
>>> from crandas.base import session
>>> session.endpoint = "https://example.com:10000/api/v1"
>>> session.base_path = Path("/cranmera")
>>> session.authorized_mode = False
>>> cd.DataFrame({})

The session can be used as a context handler to temporarily change the global session variable cd.base.session, e.g.:

session1 = cd.base.connect("session1")
session2 = cd.base.connect("session2")

cd.DataFrame() # happens in session2
cd.DataFrame(session=session1) # happens in session1

with session1:
    cd.DataFrame() # happens in session1
    cd.DataFrame(session=session2) # happens in session2
Raises:

  • ServerError – Passes through error from VDL and appropriate error code

  • ServerError – Passes through error from VDL in case error code is not known

absolute_certificate_path()

The path to the VDL endpoint certificate, as an absolute path.

property analyst_key: SigningKey | None

The (secret) signing key that the analyst that executes a query will use to sign its query. Approvers will incorporate the corresponding public key into their query approval, which ensures that only the designated analyst will be able to execute the query.

It may be set to either a crandas.sign.SigningKey instance, or a Path (or string path) pointing to such a file, or None to load the default key. Paths, or string paths containing a slash (“/”), are treated as relative to the current working directory. String paths not containing a slash are assumed to refer to a filename inside of the configuration folder (using a connection file) or the base_path (legacy mode, for backwards compatbility).

If no analyst_key is specified, the default behavior is to load analyst.sk in the configuration folder when using a connection file, or clientsign.sk in the base_path in legacy mode, for backwards compatibility reasons.

property authorized_mode

Returns whether to use authorized mode

property base_path: Path | None

The base ‘cranmera’ folder. When None, the application tries to find a parent of the current directory called ‘cranmera’, or defaults to the current directory. This is only used in legacy mode, see settings_mode.

clear_authorizations()

Clear loaded authorizations

connect(connection_file=None, mode=None)

Establishes a connection with the query server and verifies whether the client and server versions are compatible.

To disable the version check, set override_version_check to True, or set the environment variable CRANDAS_OVERRIDE_VERSION_CHECK=1.

The connect function can only be called once on a Session object. To re-connect, use the free function crandas.base.connect().

Parameters:

  • connection_file (str | Path | file-like object | True) –

    A (path to a) VDL connection file.

    It may be specified as string not containing slashes (/), backslashes () or dots (.), in which case it is interpreted as a “{name}.vdlconn” file located in the configuration folder. By default, the configuration folder is ~/.config/crandas – this can also be modified by some environment variables, see crandas.config.

    If it is a string that contains slashes, backslashes or dots, it is assumed to be a path, either a relative path for the current working directory, or an absolute path.

    If it is True, load the default connection file.

  • mode (str (optional)) – Specifies the mode (‘design’ or ‘authorized’) in case the connection file does not do so

delete(url_suffix, **kwargs)

Perform DELETE request at VDL server

Note: DELETE queries are retried via urllib, see CustomHTTPAdapter.

Parameters:

  • url_suffix (string) – URL suffix; if non-empty, should have leading slash

  • **kwargs (additional arguments to requests.Session.delete)

Returns:

result of query

Return type:

Requests.response

property design_mode

Returns whether to use design mode

property endpoint

The endpoint URL of the VDL server. This is only used in the legacy mode where the endpoint and all certificates and server public keys are specified manually. See Session for details.

get(url_suffix, **kwargs)

Perform GET request at VDL server

Note: GET queries are retried via urllib, see CustomHTTPAdapter.

Parameters:

  • url_suffix (string) – URL suffix; if non-empty, should have leading slash

  • **kwargs (additional arguments to requests.Session.get)

Returns:

result of query

Return type:

Requests.response

property keepalive

Defines whether to use HTTP keep-alive. Defaults to True

Disabling HTTP keep-alive has been found to solve issues with broken server connections in some cases. This property is set by the connection file and can be overriden in the connect function:

session = crandas.connect(…, keepalive=False)

load_authorizations(path)

Load authorizations from a file. Preserves previously loaded authorizations

property mode

Returns mode ("design" or "authorized")

post(url_suffix, **kwargs)

Perform POST request at VDL server

Data to be posted can be specified via the data kwarg.

Parameters:

  • url_suffix (string) – URL suffix; if non-empty, should have leading slash

  • **kwargs (additional arguments to requests.Session.post)

Returns:

result of query

Return type:

Requests.response

settings_mode: SettingsMode = 0

Specifies whether the Session object uses a connection file (recommended, new default) to establish a connection, or whether the endpoint is specified manually (legacy approach). In the legacy approach, all key material is found in files specified relative to the Session.base_path.

enum crandas.base.SettingsMode(value)

Bases: Enum

An enumeration.

Valid values are as follows:

CONNECTION_FILE = <SettingsMode.CONNECTION_FILE: 0>
LEGACY = <SettingsMode.LEGACY: 1>
crandas.base.connect(connection_file=None, *, set_global_session=True, mode=None, **session_args)

Connect to the VDL using a VDL connection file.

Creates a new Session with the specified arguments and then calls Session.connect().

Note: this function is also available as import crandas as cd; cd.connect().

Parameters:

  • connection_file – Connection file to use, see Session.connect()

  • set_global_session (bool, default True) – Specifies whether to overwrite the global session variable crandas.base.session

  • mode (str, optional) – Sets the mode (‘design’ or ‘authorized’) in case the connection file does not do so.

  • **session_args – Arguments to Session.__init__()

Returns:

The newly created Session object.

Return type:

Session