Managed Service for Greenplum® API, REST: PXFDatasource methods
JSON Representation
{
"name": "string",
// includes only one of the fields `s3`, `jdbc`, `hdfs`, `hive`
"s3": {
"accessKey": "string",
"secretKey": "string",
"fastUpload": true,
"endpoint": "string"
},
"jdbc": {
"driver": "string",
"url": "string",
"user": "string",
"password": "string",
"statementBatchSize": "integer",
"statementFetchSize": "integer",
"statementQueryTimeout": "integer",
"poolEnabled": true,
"poolMaximumSize": "integer",
"poolConnectionTimeout": "integer",
"poolIdleTimeout": "integer",
"poolMinimumIdle": "integer"
},
"hdfs": {
"core": {
"defaultFs": "string",
"securityAuthToLocal": "string"
},
"kerberos": {
"enable": true,
"primary": "string",
"realm": "string",
"kdcServers": [
"string"
],
"adminServer": "string",
"defaultDomain": "string",
"keytabBase64": "string"
},
"userImpersonation": true,
"username": "string",
"saslConnectionRetries": "integer",
"zkHosts": [
"string"
],
"dfs": {
"haAutomaticFailoverEnabled": true,
"blockAccessTokenEnabled": true,
"useDatanodeHostname": true,
"namenodes": "object",
"nameservices": "string"
},
"yarn": {
"resourcemanagerHaEnabled": true,
"resourcemanagerHaAutoFailoverEnabled": true,
"resourcemanagerHaAutoFailoverEmbedded": true,
"resourcemanagerClusterId": "string",
"haRm": "object"
}
},
"hive": {
"core": {
"defaultFs": "string",
"securityAuthToLocal": "string"
},
"kerberos": {
"enable": true,
"primary": "string",
"realm": "string",
"kdcServers": [
"string"
],
"adminServer": "string",
"defaultDomain": "string",
"keytabBase64": "string"
},
"userImpersonation": true,
"username": "string",
"saslConnectionRetries": "integer",
"zkHosts": [
"string"
],
"ppd": true,
"metastoreUris": [
"string"
],
"metastoreKerberosPrincipal": "string",
"authKerberosPrincipal": "string"
},
// end of the list of possible fields
}
Field | Description |
---|---|
name | string Required. Data source name. The string length in characters must be 3-200. Value must match the regular expression |
s3 | object Settings of an external S3 data source. includes only one of the fields s3 , jdbc , hdfs , hive |
s3. accessKey |
string Public key to access S3 storage. The maximum string length in characters is 200. |
s3. secretKey |
string Secret key to access S3 storage. The maximum string length in characters is 200. |
s3. fastUpload |
boolean (boolean) Manages a fast upload of big files to S3 storage. In case of the The fast upload is enabled by default. |
s3. endpoint |
string S3 storage address. The default value is The maximum string length in characters is 200. Value must match the regular expression |
jdbc | object Settings of an external JDBC data source. includes only one of the fields s3 , jdbc , hdfs , hive |
jdbc. driver |
string JDBC driver class in Java. The possible values are the following:
The maximum string length in characters is 50. |
jdbc. url |
string URL that the JDBC driver uses to connect to the database. Examples:
The maximum string length in characters is 1000. |
jdbc. user |
string Username of the DB owner. The maximum string length in characters is 200. |
jdbc. password |
string Password of the DB owner. The maximum string length in characters is 200. |
jdbc. statementBatchSize |
integer (int64) Number of rows to read in an external table, in a batch. The default value is Acceptable values are 50 to 1000, inclusive. |
jdbc. statementFetchSize |
integer (int64) Number of rows to fetch (buffer) when reading from an external table. The default value is Acceptable values are 50 to 10000, inclusive. |
jdbc. statementQueryTimeout |
integer (int64) Amount of time (in seconds) the JDBC driver waits for a statement to run. This timeout applies to statements created for both read and write operations. The default value is Acceptable values are 5 to 43200, inclusive. |
jdbc. poolEnabled |
boolean (boolean) Determines whether JDBC connection pooling is used in a server configuration. By default, it is used. |
jdbc. poolMaximumSize |
integer (int64) Maximum number of connections to the DB backend. The default value is Acceptable values are 1 to 200, inclusive. |
jdbc. poolConnectionTimeout |
integer (int64) Maximum amount of time, in milliseconds, after which an inactive connection is considered idle. The default value is Acceptable values are 5000 to 600000, inclusive. |
jdbc. poolIdleTimeout |
integer (int64) Maximum amount of time, in milliseconds, after which an inactive connection is considered idle. The default value is Acceptable values are 5000 to 600000, inclusive. |
jdbc. poolMinimumIdle |
integer (int64) Minimum number of idle connections maintained in the connection pool. The default value is Acceptable values are 0 to 200, inclusive. |
hdfs | object Settings of an external HDFS data source. includes only one of the fields s3 , jdbc , hdfs , hive |
hdfs. core |
object Settings of the file system and security rules. |
hdfs. core. defaultFs |
string URI whose scheme and authority determine the file system implementation. The maximum string length in characters is 200. |
hdfs. core. securityAuthToLocal |
string Rules for mapping Kerberos principals to operating system user accounts. The maximum string length in characters is 1000. |
hdfs. kerberos |
object Settings of the Kerberos network authentication protocol. |
hdfs. kerberos. enable |
boolean (boolean) Determines whether the Kerberos authentication server is used. By default, it is not used. |
hdfs. kerberos. primary |
string Host of the primary KDC server (Key Distribution Center). The maximum string length in characters is 128. |
hdfs. kerberos. realm |
string Kerberos realm for a Greenplum® DB. The maximum string length in characters is 1000. |
hdfs. kerberos. kdcServers[] |
string KDC server hosts. The maximum number of elements is 200. The string length in characters for each value must be 1-200. |
hdfs. kerberos. adminServer |
string Administration server host. Usually, this is the primary Kerberos server. The maximum string length in characters is 128. |
hdfs. kerberos. defaultDomain |
string Domain that is used for the host name extension. Applicable when Kerberos 4 service members become Kerberos 5 service members (for example, when rcmd.hostname is replaced with host/hostname.domain). The maximum string length in characters is 128. |
hdfs. kerberos. keytabBase64 |
string Base64 encoded contents of the keytab file. The maximum string length in characters is 65536. Value must match the regular expression |
hdfs. userImpersonation |
boolean (boolean) Enables authentication on behalf of the Greenplum® user when connecting to the remote file storage or DBMS. The authentication is disabled by default. |
hdfs. username |
string Login username for the remote file storage or DBMS if authentication on behalf of the Greenplum® user is enabled. The maximum string length in characters is 128. |
hdfs. saslConnectionRetries |
integer (int64) Maximum number of times that PXF retries a SASL connection request after a refused connection returns a The default value is Acceptable values are 1 to 50, inclusive. |
hdfs. zkHosts[] |
string ZooKeeper server hosts. Specify values in the The maximum number of elements is 200. The string length in characters for each value must be 1-200. |
hdfs. dfs |
object Settings of the distributed file system. |
hdfs. dfs. haAutomaticFailoverEnabled |
boolean (boolean) Determines whether automatic failover is enabled for the high availability of the file system. The automatic failover is enabled by default. |
hdfs. dfs. blockAccessTokenEnabled |
boolean (boolean) If The check of access tokens is enabled by default. |
hdfs. dfs. useDatanodeHostname |
boolean (boolean) Determines whether the datanode hostname is used when connecting to datanodes. |
hdfs. dfs. namenodes |
object List of HDFS service logical names. Specify them separated by commas. The names can be arbitrary. No more than 10000 per resource. |
hdfs. dfs. nameservices |
string Corresponds well-known HDFS client setting "dfs.nameservices" for this datasource |
hdfs. yarn |
object Settings of the ResourceManager service that is responsible for tracking resources in a cluster and scheduling applications (e.g., MapReduce jobs). |
hdfs. yarn. resourcemanagerHaEnabled |
boolean (boolean) Determines whether high availability is enabled for YARN's ResourceManager services. The high availability is enabled by default. |
hdfs. yarn. resourcemanagerHaAutoFailoverEnabled |
boolean (boolean) Determines whether another ResourceManager should automatically become active when the active ResourceManager has failed and does not respond. The switch of ResourceManagers is enabled by default if the high availability is enabled. |
hdfs. yarn. resourcemanagerHaAutoFailoverEmbedded |
boolean (boolean) Determines whether the embedded ActiveStandbyElector method should be used for the election of the active ResourceManager. If the current active ResourceManager has failed and does not respond, the ActiveStandbyElector method makes another ResourceManager active which then takes over. |
hdfs. yarn. resourcemanagerClusterId |
string Cluster ID. Specify it, so the ResourceManager service does not become active for a different cluster. The maximum string length in characters is 1000. |
hdfs. yarn. haRm |
object Highly available ResourceManager service. No more than 10000 per resource. |
hive | object Settings of an external Hive data source. includes only one of the fields s3 , jdbc , hdfs , hive |
hive. core |
object Settings of the file system and security rules. |
hive. core. defaultFs |
string URI whose scheme and authority determine the file system implementation. The maximum string length in characters is 200. |
hive. core. securityAuthToLocal |
string Rules for mapping Kerberos principals to operating system user accounts. The maximum string length in characters is 1000. |
hive. kerberos |
object Settings of the Kerberos network authentication protocol. |
hive. kerberos. enable |
boolean (boolean) Determines whether the Kerberos authentication server is used. By default, it is not used. |
hive. kerberos. primary |
string Host of the primary KDC server (Key Distribution Center). The maximum string length in characters is 128. |
hive. kerberos. realm |
string Kerberos realm for a Greenplum® DB. The maximum string length in characters is 1000. |
hive. kerberos. kdcServers[] |
string KDC server hosts. The maximum number of elements is 200. The string length in characters for each value must be 1-200. |
hive. kerberos. adminServer |
string Administration server host. Usually, this is the primary Kerberos server. The maximum string length in characters is 128. |
hive. kerberos. defaultDomain |
string Domain that is used for the host name extension. Applicable when Kerberos 4 service members become Kerberos 5 service members (for example, when rcmd.hostname is replaced with host/hostname.domain). The maximum string length in characters is 128. |
hive. kerberos. keytabBase64 |
string Base64 encoded contents of the keytab file. The maximum string length in characters is 65536. Value must match the regular expression |
hive. userImpersonation |
boolean (boolean) Enables authentication on behalf of the Greenplum® user when connecting to the remote file storage or DBMS. The authentication is disabled by default. |
hive. username |
string Login username for the remote file storage or DBMS if authentication on behalf of the Greenplum® user is enabled. The maximum string length in characters is 128. |
hive. saslConnectionRetries |
integer (int64) Maximum number of times that PXF retries a SASL connection request after a refused connection returns a The default value is Acceptable values are 1 to 50, inclusive. |
hive. zkHosts[] |
string ZooKeeper server hosts. Specify values in the The maximum number of elements is 200. The string length in characters for each value must be 1-200. |
hive. ppd |
boolean (boolean) Specifies if predicate pushdown is enabled for queries on external tables. The predicate pushdown is enabled by default. |
hive. metastoreUris[] |
string List of URIs separated by commas. To request metadata, the remote DBMS connects to Metastore by one of these URIs. The maximum number of elements is 200. The string length in characters for each value must be 1-200. |
hive. metastoreKerberosPrincipal |
string Service principal for the Metastore Thrift server. The maximum string length in characters is 1000. |
hive. authKerberosPrincipal |
string Kerberos server principal. The maximum string length in characters is 1000. |
Methods
Method | Description |
---|---|
create | Creates PXF datasource |
delete | Delete PXF datasource |
list | List all PXF datasources |
update | Update PXF datasource |