Data Masking
GBase 8a MPP Cluster provides a new feature of dynamic data masking, which
allows developers or database administrators to control the exposure level of
sensitive data and generate data at the database level, greatly simplifying the
security design and coding of the database application layer.
According to Permissions and Field Attributes
Users can add masking properties to the fields that need to be masked through SQL
syntax, and determine whether to expose the original data to users who have query
requirements through user permission control.
Built-in Rules
Dynamic data masking does not actually modify the actual data stored in the table,
but applies this feature to control the data returned in the query. Dynamic data
masking supports four data masking functions, including default, random, partial,
SHA, and keymask. Whether dynamic data masking is enabled depends on the
current user's permissions. Users with unmask permissions can access the actual data
without being affected by the masking rules, while users without unmask
permissions can only access the masked data affected by the masking rules. Masking
only applies to projected columns.
GBase 8a MPP Cluster Technical White Paper
General Data Technology Co., Ltd.
- 27 -
Full-Text Search
GBase 8a MPP Cluster database supports full-text search, which uses a single-word
index by default and supports almost all languages, ensuring a 100% query recall
rate. Combined with GBase 8a MPP Cluster's unique column storage, compression,
and intelligent indexing technologies, it is suitable for retrieval and query
applications aimed at massive data.
It supports establishing full-text indexes for external storage documents and
provides retrieval. The supported document formats include txt, doc, XML, HTML,
PDF, xls, and others.
The main functions include:
1.
Index Creation and Search
GBase 8a MPP Cluster has an embedded full-text search engine that
supports the indexing and querying of all text type fields in a table.
It supports parameterized management, and the processes of index
creation,
segmentation,
index
maintenance,
and
search
can
be
conveniently configured through GBase 8a MPP Cluster's standard
configuration files.
GBase 8a MPP Cluster has an embedded text tokenizer function to
achieve single-word segmentation of text columns and search strings,
ensuring consistency between their segmentation rules and results and
preventing inconsistent segmentation caused by context and language
usage.
It supports synchronous full-text index queries, enabling querying during
the index creation process. New data can be indexed in batches, and when
the indexed data in the buffer is processed and written to the index file,
GBase 8a MPP Cluster Technical White Paper
- 28 -
General Data Technology Co., Ltd.
users can immediately search for this new content that has been indexed,
instead of waiting for all new data to be indexed before querying.
It supports logical expression queries (AND, OR, NOT), NEAR queries,
and logical combination queries between full-text indexed columns and
non-full-text indexed fields in a database table.
2.
Support DML
It supports online deletion of character data type columns that have
established full-text indexes in a database table.
It supports synchronously updating full-text indexes after a column data is
updated.
3.
Support DDL
It supports automatic invalidation of indexes after a database table's
full-text indexed column is deleted.
It supports index persistence after a database table is renamed.
Library for In-Database Mining
GBMLLib is an extension library for data mining and machine learning of GBase 8a
MPP Cluster. It is added to GBase 8a MPP Cluster as a plugin. Through the machine
learning algorithms provided by GBMLLib, GBase 8a MPP Cluster can conduct
in-depth analysis and mining of user data and transform user data into user value.
GBMLLib provides SQL-based machine learning algorithms, currently including
regression
algorithms
(linear
regression),
classification
algorithms
(logistic
regression, support vector machine), and clustering algorithms (K-means). It also
provides some basic functions for array operations and linear algebra calculations.
GBMLLib has the following technical features:
GBase 8a MPP Cluster Technical White Paper
General Data Technology Co., Ltd.
- 29 -
SQL interface: GBMLLib provides SQL-based data mining algorithms,
and the training, evaluation, and prediction of models are all executed
through SQL statements, making it very easy for data analysts to master
and combine with their existing skills to fully unleash their creativity and
improve work efficiency.
In-database analysis: Unlike other analysis tools that require data to be
moved from the database to the analysis node for processing through APIs
or ODBC, GBMLLib's analysis algorithms run in the form of database
UDF/UDAF inside the threads of GBase 8a MPP Cluster, and are
scheduled through GBase 8a MPP Cluster's execution plan to minimize
data movement and improve execution speed.
Easy to extend: GBMLLib is added to GBase 8a MPP Cluster as a plugin
and adopts an elastic and flexible software architecture, making it easy to
add new data mining and machine learning algorithms in the future.
GBase 8a MPP Cluster Technical White Paper
- 30 -
General Data Technology Co., Ltd.
5 GBase8aMPPClusterOperating
EnvironmentandTechnicalIndicators
Hardware Environment
The following hardware environments are supported:
1)
Supported server platforms: x86_64 standard PC servers, PowerLinux
servers, Inspur K1, Huawei TaiShan, Sugon, Hygon, etc.;
2)
Supported storage: Local storage (SATA, SAS, SSD, etc.), array storage
(SAN, NAS), SSD, flash cards;
3)
Supported network environments: Gigabit, 10 Gigabit, InfiniBand.
Operating System and Platform
The following operating systems and platforms are supported:
1)
Supported CPUs: Intel, AMD, Shenwei, Loongson 3B, Feiteng, X86,
Power, Kunpeng, Haiguang, etc.;
2)
Supported operating systems: 64-bit operating systems such as CentOS,
Red Hat, SUSE, NeoKylin, PowerLinux, Deepin, KylinOS, LINX,
NFS-China, i-soft, etc.;
3)
Support for virtualization technologies based on x86 and Power, such as
VMware ESX and KVM, OpenStack, Docker, etc.
Interfaces with Third-Party Software Platforms
Compatible with mainstream application software, including:
1)
BI tools: MSTR, Cognos, R3 Query, SAS, SPSS, etc.;
2)
ETL tools: Datastage, Kettle, Informatica, etc.;
GBase 8a MPP Cluster Technical White Paper
General Data Technology Co., Ltd.
- 31 -
3)
Middleware: WebSphere, Tuxedo, Hibernate, Weblogic, Tomcat, etc.
Technical Specifications
Technical Secification
Description
Single node data processing capacity
100TB
Numeric precision
65
Number of tables
Limited by the file system
Ext3 file system has an upper limit of 65535
databases per single instance, while other
file systems have no such limit.
Number of columns in each table
2000
Number of rows in each table
140737488355328
Internal length of a row in a table
65534000 bytes
Length of an INTEGER type column
4 bytes
Number of bits representing the year
in a date type column
4 bits
Number of characters in a username
16 characters
Length of a CHAR type column
255 characters
Length of a BLOB column
32K bytes
Length of a VARCHAR type column
Varies with the character set
For UTF8MB4 and GB18030, is 8192, for
GBK and UTF-8, is 10922 characters.
Length of a rowstore column
32KB
Length of a virtual cluster name
255
Number of virtual clusters
64
Length of a database name
48 characters
Length of a table name
56 characters
Length of a column name
64 characters
Length of an index name
64 characters
Length of an alias
255 characters
Encoding format
UTF-8, UTF8MB4, GBK , GB18030
GBase 8a MPP Cluster Technical White Paper
- 32 -
General Data Technology Co., Ltd.
6 GBase 8a MPP Cluster Management Tools
GBase 8a MPP Cluster provides tools such as GBaseDataStudio, cluster monitoring
tools, gcadmin command-line tools, DB-Link, and transparent gateways to centrally
and uniformly manage the various functions of the database, monitor the system's
operating status, resource usage, and task execution, and achieve multiple functions
such as fast data loading and efficient data access between clusters.
Client Tools