Data structures in crandas

In crandas, there are a number of data structures that you need to know in order to get started. The two fundamental ones that you should be familiar with are Series and DataFrames. These are very similar to the Series and DataFrame objects that you find in pandas too, so they may be familiar to you already.

A DataFrame represents a table and is made out of Series that represent the columns. They work similarly to the pandas DataFrame and Series respectively, although they are structurally very different. These classes allow you to interact with the tables that are stored in the engine, but do not actually contain any data (in fact, they're not even actual tables!). Their values are remotely stored in secret-sharedform across a number of servers.

DataFrame

A DataFrame represents a table and allows you to interact with the table stored in the engine as if it were a python object. For most cases, you can treat it as if it simply were a table. For example, if we have the name of a column, we can use standard bracket notation to access it:

import crandas as cd

df = cd.DataFrame({'A': [1, 2], 'B': [3, 4]})
colA = df['A']

There are some important differences. As expected, attempting to print a DataFrame will not print the table, but just some of the metadata that crandas needs to keep track of the table

>>> print(df)
7C7621642C15C81A52D6DA9562E3C5F8...[2x2]

Privacy

DataFrames do not have an index to prevent information leakage.

Series

A Series is an object representing a column or a function applied to one or more table columns. Crandas processes operations in a lazy way, only communicating with the server whenever an output is needed. Some operations like a filter only work on a DataFrame and not on a Series For these cases, a Series can be converted to a DataFrame using Series.as_table().

df = cd.DataFrame({'A': [1, 2], 'B': [3, 4]})

series1 = df["A"]
series2 = df["A"] + 1

>>> print(series1)
    <crandas.dataframe.SeriesColRef object at 0x7fa2cc1dec80>
>>> print(series2)
    <crandas.dataframe.SeriesFun object at 0x7fa2aa4505e0>
>>> print(series1.open())
    0  1
    1  2
# A column name can be added when using as_table()
>>> print(series2.as_table(column_name="A+1").open())
        A+1
    0    2
    1    3

The names SeriesFun and SeriesColRef refer to types of Series the former referring to a function and the latter to a column reference.

Columns

When accessing a column of a DataFrame we receive a SeriesColRef. This crandas object contains additional information, like the type of the column, as well as the index of that column in that table. These are accessed by .ix() and .type() respectively.

>>> df["B"].ix() # Show the index of column "B".
1
>>> df["B"].type()
Col("B", type="integer", elperv=1)

Where the integer in Col("B", type="integer", elperv=1) refers to the fact that the column contains integer values. For more information on this, you can look at this section.

In the next section, we will see how to do simple tabular manipulations of our DataFrames.