Conversion Data Issues

StevanEarl — Fri, 11 Sep 2026 03:06:35 GMT

Problem #136 Some GROOM_SCANS direction codes have trailing spaces

Conversion Data Issues

IanGilby — Fri, 04 Sep 2026 22:36:17 GMT

* (#86) There are numerous instances in the food_part_lookup that that conflate name, initial, and/or the english translation.

@@ Line 2,286: / Line 2,286: @@
 ICG AND ELO CONFIRM 7/3/26
-addressed with commit efa75af037a5e22b2b43675b435f472193a6672f
+addressed with commits efa75af037a5e22b2b43675b435f472193a6672f and 921b5ca3e9ca3d1e06fe189b4775aae7dcc484bf
 note that the solution to problem #74 was an umbrella fix that also addressed #65-68, and #72
@@ Line 5,105: / Line 5,105: @@
 Do not load pg_year into another production column or silently replace pg_date. Retain pg_year in clean.pantgrunt_event for investigator review. The pantgrunt loader explicitly documents that the field is omitted pending a project decision about the 178 discrepancies.
+== (#136) Some GROOM_SCANS direction codes have trailing spaces ==
+=== Problem ===
+There are 3 GROOM_SCANS records where GS_direction is 'G ' instead of 'G'. The trailing Access padding prevents an exact match with the valid one-character direction code.
+=== Bad Data ===
+Cleaned in the clean schema, so query the easy schema.
+<pre>
+SELECT '"' || gs_direction || '"' AS untrimmed_direction,
+       '"' || BTRIM(gs_direction) || '"' AS trimmed_direction,
+       count(*) AS row_count
+  FROM easy.groom_scans
+ WHERE gs_direction IS DISTINCT FROM BTRIM(gs_direction)
+ GROUP BY gs_direction,
+          BTRIM(gs_direction)
+ ORDER BY gs_direction;
+</pre>
+This reports 3 rows having "G ", which normalizes to "G".
+=== Solution ===
+Trim leading and trailing whitespace from GS_direction in the clean schema. This preserves all source rows and allows the direction codes to be mapped normally by the production loader.
+Resolved by commit 3ad4f5e1e2355895e5e465c4b8049197c8dce295

@@ Line 1,472: / Line 1,472: @@
 Also: Tiki_Mom
-== * (#52) There are follow_arrivals that are almost duplicates  ==
+== (#52) There are follow_arrivals that are almost duplicates  ==
 === Problem ===
@@ Line 1,543: / Line 1,543: @@
 === Solution ===
+IAN FIXED IN ACCESS 8/24/2016
-IAN TO FIX
-Where both start and end-times are duplicated, along with other data like cycle state, I can add additional observers or data sources.  But I don't know what to do with "almost the same" data.
-FOR NOW, PROMPT A WARNING.
-==== Karl's comment ====
-I leave it to all of you to decide just what the warning(s) are to report.
 == (#53) There are follow_arrivals where the arriving chimp does not exist ==
@@ Line 2,083: / Line 2,076: @@
 === SOLUTION ===
+IAN FIXED IN ACCESS 8/25/2026
-IAN
 == (#72) There are AGGRESSION_EVENT recipient certainty flags other than `Y` or `N` (required). ==
@@ Line 2,897: / Line 2,890: @@
 === SOLUTION ===
 ALLOW FOR NOW, BUT NEEDS TO BE DISCUSSED AS A PI GROUP
+/3/2026 - ICG: Actually this returns cases where a scientific food name is associated with more than one local food name. That is, There can be multiple words for the same latin name.
 ==== Karl's comment ====
@@ Line 2,902: / Line 2,897: @@
 To allow, do some hackery when constructing the support tables.  Make non-unique values unique by adding extra "duplicate #1 --" sort of text into the value of the column.  Either "by hand" in the conversion code or by way of some algorythm.
-== * (#86) There are numerous instances in the food_part_lookup that that conflate name, initial, and/or the english translation. ==
+== (#86) There are numerous instances in the food_part_lookup that that conflate name, initial, and/or the english translation. ==
 === Problem ===
@@ Line 3,049: / Line 3,044: @@
 === SOLUTION ===
+/4/2026
-ALLOW FOR NOW. IAN/PI GROUP TO DECIDE ON PROTOCOL
+Ian fixed mifupa, miti and mizizi in FOOD_BOUT by changing to UNRECORDED. DELETED FROM FOOD_PART_LOOKUP
+Consolidated insects to 'dudu'
+changed all "NA" to "None"
+Kept unrecorded
+fixed spellings of utomvi and chipukizi
+I made all associated changes in FOOD_BOUT, choosing to use names rather than initials
 == * (#87) There are numerous duplicate fl_sci_food_name_gen values in food_lookup. ==

sokwedb - Recent changes [en]

Conversion Data Issues

Conversion Data Issues