Sunday, 7 October 2018

MYSQL SYS SCHEMA IN MARIADB 10.2


MySQL has introduced the PERFORMANCE_SCHEMA (P_S) in MySQL 5.5 and made it really usable in MySQL 5.6 and added some enhancements in MySQL 5.7 and 8.0.

Unfortunately the PERFORMANCE_SCHEMA was not really intuitive or effective for the broader audience and didn’t provide the deep details easily. Therefore Mark Leith introduced the sys Schema for an easier access for the Developer, DBA and DevOps and Then Daniel Fischer has enhanced it further. sys schema is a set of objects that helps DBAs and developers interpret data collected by the Performance Schema. sys schema objects can be used for typical tuning and diagnosis use cases. Objects in this schema include:

  1. Views that summarize Performance Schema data into more easily understandable form.
  2. Stored procedures that perform operations such as Performance Schema configuration and generating diagnostic reports.
  3. Stored functions that query Performance Schema configuration and provide formatting services.

For latest installations, The sys Schema is installed by default during data directory initialization and is not comes separately for now.

Fortunately the sys Schema up to version 1.5.1 is available on GitHub. So we can adapt and use it for MariaDB as well. The version of the sys Schema in MySQL 8.0 is 1.6.0 and seems not to be on GitHub yet. But you can extract it from the MySQL 8.0 directory structure: mysql-8.0/share/mysql_sys_schema.sql.  The GitHub developers are just been working on other priorities. and I believe they will announced another release soon (I think they are working on it at the moment).

MariaDB has integrated the PERFORMANCE_SCHEMA based on MySQL 5.6 into its own MariaDB 10.2 server but they did not integrate the sys Schema. Which PERFORMANCE_SCHEMA version is integrated in MariaDB can be found By this Link.

To install the sys Schema into MariaDB we first have to check if the PERFORMANCE_SCHEMA is activated in the MariaDB server:

mariadb> SHOW GLOBAL VARIABLES LIKE 'performance_schema';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| performance_schema | OFF   |
+--------------------+-------+

To enable the PERFORMANCE_SCHEMA just add the following line to your my.cnf:

[mysqld]
performance_schema = 1

and restart the Services.
If restarting mysql service gives an error about performance schema then you can add below code to my.cnf file it will work the same way.
[mysqld]
performance_schema
performance_schema_events_statements_history_long_size=10000

If you don't know on the location of My.cnf, the default location is /etc/mysql/my.cnf, for RHEL Servers it will located at /etc/my.cnf, for windows server it is my.ini and it will located at (mysql\bin\my.ini
if are using some other servers or unable to find it you can just search for it on the internet.

In MariaDB 10.2 the MySQL 5.6 PERFORMANCE_SCHEMA is integrated so we can not to Install sys_57.sql Instead of it we have to run the sys_56.sql installation script. If you try to run the sys_57.sql script you will get a lot of errors which are very difficult to solve.

But also the sys_56.sql installation script also have some little troubles which but these are are easy to fix:

unzip mysql-sys-1.5.1.zip mysql -uroot < sys_56.sql
ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable '
ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable 'master_info_repository'
ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable 'relay_log_info_repository'

For a quick fix and make the sys Schema work I changed the following information:
  1. server_uuid to server_id
  2. master_info_repository to NULL(just replace it 3 times)
  3. relay_log_info_repository to NULL (just replace it 3 times)

For the future the DB community has to think about if the sys Schema should be aware of the 2 branches MariaDB and MySQL and act accordingly or if the sys Schema has to be forked to work properly for MariaDB and implement MariaDB specific functionality.

When the sys Schema finally is installed properly. you have the following tables to get your performance metrics:

mariadb> use sys
mariadb> SHOW TABLES;
+-----------------------------------------------+
| Tables_in_sys                                 |
+-----------------------------------------------+
| host_summary                                  |
| host_summary_by_file_io                       |
| host_summary_by_file_io_type                  |
| host_summary_by_stages                        |
| host_summary_by_statement_latency             |
| host_summary_by_statement_type                |
| innodb_buffer_stats_by_schema                 |
| innodb_buffer_stats_by_table                  |
| innodb_lock_waits                             |
| io_by_thread_by_latency                       |
| io_global_by_file_by_bytes                    |
| io_global_by_file_by_latency                  |
| io_global_by_wait_by_bytes                    |
| io_global_by_wait_by_latency                  |
| latest_file_io                                |
| metrics                                       |
| processlist                                   |
| ps_check_lost_instrumentation                 |
| schema_auto_increment_columns                 |
| schema_index_statistics                       |
| schema_object_overview                        |
| schema_redundant_indexes                      |
| schema_table_statistics                       |
| schema_table_statistics_with_buffer           |
| schema_tables_with_full_table_scans           |
| schema_unused_indexes                         |
| session                                       |
| statement_analysis                            |
| statements_with_errors_or_warnings            |
| statements_with_full_table_scans              |
| statements_with_runtimes_in_95th_percentile   |
| statements_with_sorting                       |
| statements_with_temp_tables                   |
| sys_config                                    |
| user_summary                                  |
| user_summary_by_file_io                       |
| user_summary_by_file_io_type                  |
| user_summary_by_stages                        |
| user_summary_by_statement_latency             |
| user_summary_by_statement_type                |
| version                                       |
| wait_classes_global_by_avg_latency            |
| wait_classes_global_by_latency                |
| waits_by_host_by_latency                      |
| waits_by_user_by_latency                      |
| waits_global_by_latency                       |
+-----------------------------------------------+






Monday, 11 September 2017

WEEK 6 :APPLICATION ENGINEERING : M101P: MONGODB FOR DEVELOPERS


Homework 6.1

Which of the following statements are true about replication in MongoDB? Check all that apply.

Solution : Here you find updated answer with two possible outcomes as follow


Homework 6.2

Let's suppose you have a five member replica set and want to assure that writes are committed to the journal and are acknowledged by at least 3 nodes before you proceed forward. What would be the appropriate settings for w and j?
Solution : 


Homework 6.3

Which of the following statements are true about choosing and using a shard key?

Solution : There are three possibly true statement for this homework.




Homework 6.4

You have a sharded system with three shards and have sharded the collections "students" in the "school" database across those shards. The output of sh.status() when connected to mongos looks like this:

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5531512ac723271f602db407")
}
  shards:
    {  "_id" : "s0",  "host" : "s0/localhost:37017,localhost:37018,localhost:37019" }
    {  "_id" : "s1",  "host" : "s1/localhost:47017,localhost:47018,localhost:47019" }
    {  "_id" : "s2",  "host" : "s2/localhost:57017,localhost:57018,localhost:57019" }
  balancer:
    Currently enabled:  yes
    Currently running:  yes
        Balancer lock taken at Fri Apr 17 2015 14:32:02 GMT-0400 (EDT) by education-iMac-2.local:27017:1429295401:16807:Balancer:1622650073
    Collections with active migrations:
        school.students started at Fri Apr 17 2015 14:32:03 GMT-0400 (EDT)
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        2 : Success
        1 : Failed with error 'migration already in progress', from s0 to s1
  databases:
    {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
    {  "_id" : "school",  "partitioned" : true,  "primary" : "s0" }
        school.students
            shard key: { "student_id" : 1 }
            chunks:
                s0  1
                s1  3
                s2  1
            { "student_id" : { "$minKey" : 1 } } -->> { "student_id" : 0 } on : s2 Timestamp(3, 0)
            { "student_id" : 0 } -->> { "student_id" : 2 } on : s0 Timestamp(3, 1)
            { "student_id" : 2 } -->> { "student_id" : 3497 } on : s1 Timestamp(3, 2)
            { "student_id" : 3497 } -->> { "student_id" : 7778 } on : s1 Timestamp(3, 3)
            { "student_id" : 7778 } -->> { "student_id" : { "$maxKey" : 1 } } on : s1 Timestamp(3, 4)


Solution : S1



Homework 6.5

In this homework you will build a small replica set on your own computer. We will check that it works with validate.py, which you should download from the Download Handout link.
Create three directories for the three mongod processes. On unix, this could be done as follows:

mkdir -p /data/rs1 /data/rs2 /data/rs3

Now start three mongo instances as follows. Note that are three commands. The browser is probably wrapping them visually.

mongod --replSet m101 --logpath "1.log" --dbpath /data/rs1 --port 27017 --smallfiles --oplogSize 64 --fork

mongod --replSet m101 --logpath "2.log" --dbpath /data/rs2 --port 27018 --smallfiles --oplogSize 64 --fork

mongod --replSet m101 --logpath "3.log" --dbpath /data/rs3 --port 27019 --smallfiles --oplogSize 64 --fork
Windows users: Omit -p from mkdir. Also omit --fork and use start mongod with Windows compatible paths (i.e. backslashes "\") for the --dbpath argument (e.g; C:\data\rs1).
Now connect to a mongo shell and make sure it comes up

Now you will create the replica set. Type the following commands into the mongo shell:

config = { _id: "m101", members:[
          { _id : 0, host : "localhost:27017"},
          { _id : 1, host : "localhost:27018"},
          { _id : 2, host : "localhost:27019"} ]
};
rs.initiate(config);
At this point, the replica set should be coming up. You can type

rs.status()
to see the state of replication.
Now run validate.py to confirm that it works.

python validate.py
Validate connects to your local replica set and checks that it has three nodes. It has been tested under Pymongo 2.3 and 2.4. Type the validation code below.

Solution :kjvjkl3290mf0m20f2kjjv


At the end I just want to say if you want to learn mongoDB, First practice with youself. because no one can steel talent 😇