destinations.sql_client

TJobQueryTags Objects

class TJobQueryTags(TypedDict)

[view_source]

Applied to sql client when a job using it starts. Using to tag queries

SqlClientBase Objects

class SqlClientBase(ABC, Generic[TNativeConn])

[view_source]

database_name

Database or catalog name, optional

dataset_name

Normalized dataset name

staging_dataset_name

Normalized staging dataset name

capabilities

Instance of adjusted destination capabilities

drop_tables

def drop_tables(*tables: str) -> None

[view_source]

Drops a set of tables if they exist

execute_fragments

def execute_fragments(fragments: Sequence[AnyStr], *args: Any,
                      **kwargs: Any) -> Optional[Sequence[Sequence[Any]]]

[view_source]

Executes several SQL fragments as efficiently as possible to prevent data copying. Default implementation just joins the strings and executes them together.

execute_many

def execute_many(statements: Sequence[str], *args: Any,
                 **kwargs: Any) -> Optional[Sequence[Sequence[Any]]]

[view_source]

Executes multiple SQL statements as efficiently as possible. When client supports multiple statements in a single query they are executed together in as few database calls as possible.

make_qualified_table_name_path

def make_qualified_table_name_path(table_name: Optional[str],
                                   escape: bool = True) -> List[str]

[view_source]

Returns a list with path components leading from catalog to table_name. Used to construct fully qualified names. table_name is optional.

get_qualified_table_names

def get_qualified_table_names(table_name: str,
                              escape: bool = True) -> Tuple[str, str]

[view_source]

Returns qualified names for table and corresponding staging table as tuple.

with_alternative_dataset_name

@contextmanager
def with_alternative_dataset_name(
        dataset_name: str) -> Iterator["SqlClientBase[TNativeConn]"]

[view_source]

Sets the dataset_name as the default dataset during the lifetime of the context. Does not modify any search paths in the existing connection.

set_query_tags

def set_query_tags(tags: TJobQueryTags) -> None

[view_source]

Sets current schema (source), resource, load_id and table name when a job starts

DBApiCursorImpl Objects

class DBApiCursorImpl(DBApiCursor)

[view_source]

A DBApi Cursor wrapper with dataframes reading functionality

df

def df(chunk_size: int = None, **kwargs: Any) -> Optional[DataFrame]

[view_source]

Fetches results as data frame in full or in specified chunks.

May use native pandas/arrow reader if available. Depending on the native implementation chunk size may vary.