Back to the ingestion log page

Ingesting a regular mysqldump file

This wikipage gives statistics related the nebulous ingestion that happened in early June 2012.

File name	Size before	Size before (MB)	Size after	Size after (MB)	Difference	Ingestion time (s)	Ingestion Rate (MB/s)	#Entries	Ingestion rate (entries/s)
cabinet.ibd	131072	0.12	131072	0.12	0%
deleted.ibd	348127232	332.00	289406976	276.00	-17%
directory.ibd	22993174528	21928.00	21583888384	20584.00	-6%	911	22.59	16895977	18547
instance.ibd	558735818752	532852.00	452494098432	431532.00	-19%	309374	1.39	1216215431	3931
lock_record.ibd	114688	0.11	114688	0.11	0%
log.ibd	98304	0.09	98304	0.09	0%
lost_instances.ibd	688128	0.66	688128	0.66	0%
mountedvol.ibd	229376	0.22	229376	0.22	0%
storage_object.ibd	493929627648	471048.00	412044230656	392956.00	-17%	49940	7.87	944739798	18917
storage_object_attr.ibd	50675580928	48328.00	37694210048	35948.00	-26%	11623	3.09	944739798	81282
storage_object_xattr.ibd	44417679360	42360.00	30274486272	28872.00	-32%	5250	5.50	274638705	52312
volume.ibd	212992	0.20	212992	0.20	0%

Ingesting binlogs

• Binlogs have a fixed size roughly 1GB
• They contain 'BEGIN' and 'COMMIT' very often. So if autocommit is set on the server, when piping the output of mysqlbinlog to a mysql client, there is not need to COMMIT.
• Statements like 'CREATE USER', 'USE <database>' should be filtered.
• A binlog in ascii form is about 40-45 millions lines long.
• Ingesting a binlog without filtering lasted about 3 hours; with filtering between 1.5 and 2 hours.