ePrivacy and GPDR Cookie Consent by Cookie Consent Skip to main content

Loader Postgres

PostgreSQL, often referred to as Postgres, is an object-relational database management system.

Postgres uses and extends the SQL language combined with many other features that safely stores and scales the most complicated data workloads. 

The native data types that are supported include: 

Primitives: Integer, Numeric, String, Boolean

Structured: Date/Time, Array, Range, UUID 

Document: JSON/JSONB, XML, Key-value (Hstore) 

Geometry: Point, Line, Circle, Polygon 

Customizations: Composite, Custom Types

Learn more about Postgres from PostgreSQL tutorials. 

The Postgres loader is used to load data into a table in the PostgreSQL database. 

 

 

Requirements

Credentials of the database you would like to connect to:

  • Host
  • Port
  • Name of the database, schema
  • Username
  • Password
  • SSH key details (optional, described in more detail below)

If you wish to connect to a database outside of the Meiro  Integrations environment, you may need a front-end tool like PgAdmin4, Postico, DBeaver or any other front-end (GUI) tool available online. To learn more about these tools, please refer to their respective documentation online. Connecting via a GUI tool is not mandatory, but it may be useful if you want to access the database outside the Meiro  Integrations environment.

Postgres components work only with CSV tables. Other data formats need to be converted to CSV format by processors.

 

PostgreSQL  documentation

 

Data In/Data Out

Data In

Data from the previous component should be saved as a CSV table in in/tables.

 

Data Out

N/A


You can read more about the folder structure here.

 

 

Parameters

DB

Loader-Postgres-DB.png

The following  credentials are mandatory:

  • Host: can be numerical (like 111.11.111.111) or alphabetical (database.client.solutions),
  • Port: usually 5432 by default.
  • Username
  • Password 
  • Database
  • Schema

You should be able to get all the credentials from the database administrator. 

 

 

SSH

Loader-Postgres-SSH-Key.png

To connect to certain databases (e.g. databases in the internal networks or databases behind firewalls), you may need an additional security layer and an SSH Key. 

After generating the public and private keys and sending the public key to your database administrator, you can connect the database to Meiro. 

 

Enabled  (true/false)

To enable the SSH Tunnel to work, you will need to check the option marked Enabled.

 

Private and Public keys (required)

You will receive your public and private keys after generating it on your computer. For more information, you may refer to this article.

 

SSH host (required)

You will get the host from the database administrator.

 

Table

Loader-POstgres-Table.png

 

Input Table (required)

Insert the file name of the table from the Data In bucket that you wish to load to the Postgres database without the extension.

 

DB Table Name (required)

Insert the name under which you wish to save the table in the Postgres database or table you wish to update. The component will create a new table or update the existing one. 

 

Incremental (optional)

Check the Incremental’box if you wish to load only the new data. Data will be loaded incrementally using the Primary Key. If the Incremental box is left unmarked, existing data will be removed and loaded again as defined in the File name.

 

Primary Key (optional)

Primary Key defines the update of the data for the incremental load. Otherwise, the Primary Key can also be defined for the full load. 

 

Column

You will need to name the columns in your database that you wish to load. The component will create columns names according to the form described below. You will need to define the fields for each column.

Column Name (required)

The name of the columns that you want to load onto the Postgres server should correspond with the name of the columns within the file name that you’ve previously input in the Table section.

 

DB Column Name (required)

The name of the column under which it will appear in the Postgres server.

 

Data Type (required)

Insert the type of data formats that your data will take on. Data formats need to be compatible with the available Postgres data types. 

 

Size (optional)

The number of characters that you wish to load on the Postgres server. 

 

Nullable (true/false)

Nullable allows null values to be saved in the column.

 

Default Value (optional)

This is used to define the default value for a column. The default value will be added to all new records.

 

Back to top