Managed Service for Greenplum® API, REST: PXFDatasource.List
List all PXF datasources
HTTP request
GET https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/{clusterId}/pxf_datasources
Path parameters
|
Field |
Description |
|
clusterId |
string Required field. |
Response
HTTP Code: 200 - OK
{
"datasources": [
{
"name": "string",
// Includes only one of the fields `s3`, `jdbc`, `hdfs`, `hive`
"s3": {
"accessKey": "string",
"secretKey": "string",
"fastUpload": "boolean",
"endpoint": "string"
},
"jdbc": {
"driver": "string",
"url": "string",
"user": "string",
"password": "string",
"statementBatchSize": "string",
"statementFetchSize": "string",
"statementQueryTimeout": "string",
"poolEnabled": "boolean",
"poolMaximumSize": "string",
"poolConnectionTimeout": "string",
"poolIdleTimeout": "string",
"poolMinimumIdle": "string"
},
"hdfs": {
"core": {
"defaultFs": "string",
"securityAuthToLocal": "string"
},
"kerberos": {
"enable": "boolean",
"primary": "string",
"realm": "string",
"kdcServers": [
"string"
],
"adminServer": "string",
"defaultDomain": "string",
"keytabBase64": "string"
},
"userImpersonation": "boolean",
"username": "string",
"saslConnectionRetries": "string",
"zkHosts": [
"string"
],
"dfs": {
"haAutomaticFailoverEnabled": "boolean",
"blockAccessTokenEnabled": "boolean",
"useDatanodeHostname": "boolean",
"namenodes": "object",
"nameservices": "string"
},
"yarn": {
"resourcemanagerHaEnabled": "boolean",
"resourcemanagerHaAutoFailoverEnabled": "boolean",
"resourcemanagerHaAutoFailoverEmbedded": "boolean",
"resourcemanagerClusterId": "string",
"haRm": "object"
}
},
"hive": {
"core": {
"defaultFs": "string",
"securityAuthToLocal": "string"
},
"kerberos": {
"enable": "boolean",
"primary": "string",
"realm": "string",
"kdcServers": [
"string"
],
"adminServer": "string",
"defaultDomain": "string",
"keytabBase64": "string"
},
"userImpersonation": "boolean",
"username": "string",
"saslConnectionRetries": "string",
"zkHosts": [
"string"
],
"ppd": "boolean",
"metastoreUris": [
"string"
],
"metastoreKerberosPrincipal": "string",
"authKerberosPrincipal": "string"
}
// end of the list of possible fields
}
]
}
|
Field |
Description |
|
datasources[] |
PXFDatasource
|
Field |
Description |
|
name |
string Required field. Data source name. |
|
s3 |
Settings of an external S3 data source. Includes only one of the fields |
|
jdbc |
Settings of an external JDBC data source. Includes only one of the fields |
|
hdfs |
Settings of an external HDFS data source. Includes only one of the fields |
|
hive |
Settings of an external Hive data source. Includes only one of the fields |
PXFDatasourceS3
|
Field |
Description |
|
accessKey |
string Public key to access S3 storage. |
|
secretKey |
string Secret key to access S3 storage. |
|
fastUpload |
boolean Manages a fast upload of big files to S3 storage. In case of the The fast upload is enabled by default. |
|
endpoint |
string S3 storage address. The default value is |
PXFDatasourceJDBC
|
Field |
Description |
|
driver |
string JDBC driver class in Java. The possible values are the following:
|
|
url |
string URL that the JDBC driver uses to connect to the database. Examples:
|
|
user |
string Username of the DB owner. |
|
password |
string Password of the DB owner. |
|
statementBatchSize |
string (int64) Number of rows to read in an external table, in a batch. The default value is |
|
statementFetchSize |
string (int64) Number of rows to fetch (buffer) when reading from an external table. The default value is |
|
statementQueryTimeout |
string (int64) Amount of time (in seconds) the JDBC driver waits for a statement to run. This timeout applies to statements created for both read and write operations. The default value is |
|
poolEnabled |
boolean Determines whether JDBC connection pooling is used in a server configuration. By default, it is used. |
|
poolMaximumSize |
string (int64) Maximum number of connections to the DB backend. The default value is |
|
poolConnectionTimeout |
string (int64) Maximum time, in milliseconds, to wait for a connection from the pool. The default value is |
|
poolIdleTimeout |
string (int64) Maximum amount of time, in milliseconds, after which an inactive connection is considered idle. The default value is |
|
poolMinimumIdle |
string (int64) Minimum number of idle connections maintained in the connection pool. The default value is |
PXFDatasourceHDFS
|
Field |
Description |
|
core |
Settings of the file system and security rules. |
|
kerberos |
Settings of the Kerberos network authentication protocol. |
|
userImpersonation |
boolean Enables authentication on behalf of the Greenplum® user when connecting to the remote file storage or DBMS. The authentication is disabled by default. |
|
username |
string Login username for the remote file storage or DBMS if authentication on behalf of the Greenplum® user is enabled. |
|
saslConnectionRetries |
string (int64) Maximum number of times that PXF retries a SASL connection request after a refused connection returns a The default value is |
|
zkHosts[] |
string ZooKeeper server hosts. Specify values in the |
|
dfs |
Settings of the distributed file system. |
|
yarn |
Settings of the ResourceManager service that is responsible for tracking resources in a cluster and scheduling applications (e.g., MapReduce jobs). |
PXFDatasourceCore
|
Field |
Description |
|
defaultFs |
string URI whose scheme and authority determine the file system implementation. |
|
securityAuthToLocal |
string Rules for mapping Kerberos principals to operating system user accounts. |
PXFDatasourceKerberos
|
Field |
Description |
|
enable |
boolean Determines whether the Kerberos authentication server is used. By default, it is not used. |
|
primary |
string Host of the primary KDC server (Key Distribution Center). |
|
realm |
string Kerberos realm for a Greenplum® DB. |
|
kdcServers[] |
string KDC server hosts. |
|
adminServer |
string Administration server host. Usually, this is the primary Kerberos server. |
|
defaultDomain |
string Domain that is used for the host name extension. Applicable when Kerberos 4 service members become Kerberos 5 service members (for example, when rcmd.hostname is replaced with host/hostname.domain). |
|
keytabBase64 |
string Base64 encoded contents of the keytab file. |
PXFDatasourceHDFSDfs
|
Field |
Description |
|
haAutomaticFailoverEnabled |
boolean Determines whether automatic failover is enabled for the high availability of the file system. The automatic failover is enabled by default. |
|
blockAccessTokenEnabled |
boolean If The check of access tokens is enabled by default. |
|
useDatanodeHostname |
boolean Determines whether the datanode hostname is used when connecting to datanodes. |
|
namenodes |
object (map<string, PXFDatasourceHDFSDfsNamenode>) List of HDFS service logical names. Specify them separated by commas. The names can be arbitrary. |
|
nameservices |
string Corresponds well-known HDFS client setting "dfs.nameservices" for this datasource |
PXFDatasourceHDFSDfsNamenode
|
Field |
Description |
|
rpcAddress |
string |
|
serviceRpcAddress |
string |
|
httpAddress |
string |
|
httpsAddress |
string |
PXFDatasourceHDFSYarn
|
Field |
Description |
|
resourcemanagerHaEnabled |
boolean Determines whether high availability is enabled for YARN's ResourceManager services. The high availability is enabled by default. |
|
resourcemanagerHaAutoFailoverEnabled |
boolean Determines whether another ResourceManager should automatically become active when the active ResourceManager has failed and does not respond. The switch of ResourceManagers is enabled by default if the high availability is enabled. |
|
resourcemanagerHaAutoFailoverEmbedded |
boolean Determines whether the embedded ActiveStandbyElector method should be used for the election of the active ResourceManager. If the current active ResourceManager has failed and does not respond, the ActiveStandbyElector method makes another ResourceManager active which then takes over. |
|
resourcemanagerClusterId |
string Cluster ID. Specify it, so the ResourceManager service does not become active for a different cluster. |
|
haRm |
object (map<string, PXFDatasourceHDFSYarnHaRm>) Highly available ResourceManager service. |
PXFDatasourceHDFSYarnHaRm
|
Field |
Description |
|
resourcemanagerAddress |
string |
|
resourcemanagerSchedulerAddress |
string |
|
resourcemanagerResourceTrackerAddress |
string |
|
resourcemanagerAdminAddress |
string |
|
resourcemanagerWebappAddress |
string |
|
resourcemanagerWebappHttpsAddress |
string |
PXFDatasourceHive
|
Field |
Description |
|
core |
Settings of the file system and security rules. |
|
kerberos |
Settings of the Kerberos network authentication protocol. |
|
userImpersonation |
boolean Enables authentication on behalf of the Greenplum® user when connecting to the remote file storage or DBMS. The authentication is disabled by default. |
|
username |
string Login username for the remote file storage or DBMS if authentication on behalf of the Greenplum® user is enabled. |
|
saslConnectionRetries |
string (int64) Maximum number of times that PXF retries a SASL connection request after a refused connection returns a The default value is |
|
zkHosts[] |
string ZooKeeper server hosts. Specify values in the |
|
ppd |
boolean Specifies if predicate pushdown is enabled for queries on external tables. The predicate pushdown is enabled by default. |
|
metastoreUris[] |
string List of URIs separated by commas. To request metadata, the remote DBMS connects to Metastore by one of these URIs. |
|
metastoreKerberosPrincipal |
string Service principal for the Metastore Thrift server. |
|
authKerberosPrincipal |
string Kerberos server principal. |