Changes between Version 6 and Version 7 of IPP_Progress_Report_20120206

Timestamp:

Feb 13, 2012, 10:14:35 AM (14 years ago)

Author:

Serge CHASTEL

Comment:

--

IPP_Progress_Report_20120206

@@ -7 +7 @@
 
 === Serge Chastel ===
- * Part of the work was done with Heather
  * Still populating ipp003 with nebulous dump (20120206).
  * Ran slocate on the hosts. All index files are in <hostname>:/tmp/slocate. Note: slocate is different from one wave to another.
- * mysql partitioning sucks (user-defined functions cannot be used for hash partitioning)
- * 
-
+ * mysql partitioning sucks (e.g. user-defined functions cannot be used for hash partitioning)
+ * Bad files hunt in nebulous and/or on ippb02 (with Heather):
+   - Ran slocate on ippb02.[0-2]/nebulous (got ~10.7M files) -> This version runs incredibly fast. Scanning all three ippb02 disks took 22 minutes (no attempt to optimize)
+   - Checked their status in nebulous: 
+     - ok
+     - uri not present in instance table for a given ins_id: ~5M files (Note: it seems that some files are temporary)
+       For a lot of those files, the uri in nebulous is file:///data/ippb02.1/nebulous[...] while the file on disk is file:///data/ippb02.0/nebulous[...].
+     - uri not present in instance table and no ins_id but "similar" uris exist: 189k files
+{{{
+Example of two similar uris:
+file:///data/ippb02.0/nebulous/00/00/1006632459.gpc1:ThreePi.nt:2011:01:23:o5584g0495o.285189:o5584g0495o.285189.ch.182968.XY34.mdl.fits
+file:///data/ipp025.0/nebulous/00/00/1230393764.gpc1:ThreePi.nt:2011:01:23:o5584g0495o.285189:o5584g0495o.285189.ch.182968.XY34.mdl.fits
+}}}
+   - No entry in nebulous but there is a file on the disk: ~349k files (e.g: gpc1/ThreePi.nt/2010/12/30/o5560g0658o.269979/o5560g0658o.269979.ch.174390.XY03.ptn)
+   Total: 5.7M "problems"
+   See /tmp/nebulous_contents/problems.
 === Heather Flewelling ===