tag:blogger.com,1999:blog-47886727817318556632024-03-05T03:40:02.351-08:00not what you were looking for Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.comBlogger47125tag:blogger.com,1999:blog-4788672781731855663.post-1952301193736193922017-10-07T13:24:00.002-07:002017-10-07T13:24:39.404-07:00Relative Time'Rel Time'<br />
<br />
Leap years don't really cause a problem -- except for 1900.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRkPUXcCmfM83Kglsd1N2rAr2ubA2xoUaT6i80D4vxGbG427VabvZQu64iOroOJYPaxDipejlYOnLhccgTbTSaf1k7cm7u98CJDzJm9qeFohoA3pmuTJ2gfYD4OJ8GNlhvtt3OyUaz4p7C/s1600/RelTimeError.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="505" data-original-width="921" height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRkPUXcCmfM83Kglsd1N2rAr2ubA2xoUaT6i80D4vxGbG427VabvZQu64iOroOJYPaxDipejlYOnLhccgTbTSaf1k7cm7u98CJDzJm9qeFohoA3pmuTJ2gfYD4OJ8GNlhvtt3OyUaz4p7C/s640/RelTimeError.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
These errors are due to the use of monthly averages where the differing number of days in each month are not accounted for.</div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-14837950301299642912017-10-06T10:52:00.003-07:002017-10-06T19:54:03.132-07:00ENSO modelingAn ENSO model has been developed by Paul Pukite and Paul has given me a copy of his spreadsheet. The spreadsheet was designed to find a best model fit to the Southern Oscillation using one of the Southern Oscillation indexes (SOI); Paul was using the NINO34 index.<br />
<br />
I decided to expand the choice of index to include all that I could find on the web. I found 10 in total. Five that stretch all the way back to 1880 and five from 1950 to the present. The five that go back to 1880 are the NINO34 index, the 'Cold Tongue Index,' the SOI_signal index, the SOI_noise index, and the SOI_boma (Bureau of Meteorology - Australia).<br />
<br />
I trained the model on the period from 1885 - 1935 using the CTI dataset. I then calculated the correlation coefficients for the Model output versus the various datasets over various time periods. Here are the results.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaET8B6W_NjlVgXoDrTgl9OwBWIWbzMEf6UxQo41N0J9VdA7d5GKhv_0chwyihgrn2ZlwPJVuDRZLWml1GJL3e1Zf_e-pqBPjnK4eJQ0zL9ZYpoJ4NXYA4eq6QPgTQJw9MS1bb1sTw0NIR/s1600/CorCoeff_1885_1935_CTI.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="708" data-original-width="527" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaET8B6W_NjlVgXoDrTgl9OwBWIWbzMEf6UxQo41N0J9VdA7d5GKhv_0chwyihgrn2ZlwPJVuDRZLWml1GJL3e1Zf_e-pqBPjnK4eJQ0zL9ZYpoJ4NXYA4eq6QPgTQJw9MS1bb1sTw0NIR/s1600/CorCoeff_1885_1935_CTI.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
This result was not wholly unexpected the different ENSO datasets are highly correlated - except for one outlier in each group.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihQRmrpzGY2SkQHxBEoYzVL2vhjLiBhrDRaWQ89LV0Ov_GepyiP2abI5WegVnfHXnZzICYYoaisa6cqnn_bDmOJmBzNzKmmnNYQWYFGWvmqjda2M1Z-G-WwAJji0lfl7SXlVP5hBLqaHSx/s1600/ENSO_indexes_CCs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="292" data-original-width="708" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihQRmrpzGY2SkQHxBEoYzVL2vhjLiBhrDRaWQ89LV0Ov_GepyiP2abI5WegVnfHXnZzICYYoaisa6cqnn_bDmOJmBzNzKmmnNYQWYFGWvmqjda2M1Z-G-WwAJji0lfl7SXlVP5hBLqaHSx/s1600/ENSO_indexes_CCs.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-12718029380876018472017-10-02T08:07:00.002-07:002017-10-02T08:07:41.195-07:00NINO34 - SOI sliding correlation<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT41a6X8eSBNJzNYsrQP5xaGJVmBcCJlfz028Jay1iWhyJO3gcSVfPO5dGRlSScrnUb5P_mIJyGJEGOokLQp8SrUQbox2dIRE4RPmUU4p0Rf8mzfxpEjA3MtuueFlilGphVMS1gLDkHMqA/s1600/NINO34+SOI+sliding+correlation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="921" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT41a6X8eSBNJzNYsrQP5xaGJVmBcCJlfz028Jay1iWhyJO3gcSVfPO5dGRlSScrnUb5P_mIJyGJEGOokLQp8SrUQbox2dIRE4RPmUU4p0Rf8mzfxpEjA3MtuueFlilGphVMS1gLDkHMqA/s1600/NINO34+SOI+sliding+correlation.png" /></a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-84122670236040648742017-10-01T12:16:00.001-07:002017-10-01T17:11:06.944-07:00Natural Variability in the GMST recordWhat is the natural variability in the surface temperature record?<br />
<br />
One way to explore this is to take the temperature record and subtract all the known non-random forcings and fluctuations. What's left over is our best guess at natural variation.<br />
<br />
I did this with the HADCRUT global land/ocean temperature dataset from 1881 thru 2010. The known forcing that I used were:<br />
<br />
<ul>
<li>CO2</li>
<li>TSI</li>
<li>LOD</li>
<li>ENSO (using two different ENSO indexes for comparison purposes)</li>
</ul>
The result looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEyL3FffQti7I58Po_Ntj7Q2QmVrRjYHeVouNQuveIbDfdexK7Xy6smFUk7AEt2dglGNj-8v-TDDCtZMtoXOk5OF8iY0piPMFY7y-tIwTQzKYKL3AngpZ1dfQRSNU6qD_lSM2_CBtP6fvh/s1600/Hadcrut+natural+variability.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="493" data-original-width="676" height="465" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEyL3FffQti7I58Po_Ntj7Q2QmVrRjYHeVouNQuveIbDfdexK7Xy6smFUk7AEt2dglGNj-8v-TDDCtZMtoXOk5OF8iY0piPMFY7y-tIwTQzKYKL3AngpZ1dfQRSNU6qD_lSM2_CBtP6fvh/s640/Hadcrut+natural+variability.png" width="640" /></a></div>
<br />
Using the NINO34 ENSO dataset the standard deviation is 0.105503°C<br />
Using the CTI ('Cold Tongue Index') the standard deviation is 0.107541°C<br />
<br />
With global warming since pre-industrial at approximately 1°C, natural variation is not the answer. It's known forcings -- and CO2 is most of that answer.<br />
<br />
We should also check the sensitivity of the analysis to the various datasets used. In the first analysis we looked at two different ENSO indexes. A second analysis replaces the HADCRUT gmst data with the BEST gmst dataset. The results are the same to within a few thousandths of a degree:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwPdpTiPKLKRIkmQ5JFjC_7AuxTaivbSPMapuDxFMHRhvRHUmFLusrndT8cRKWBdGbnSb9-Bg_DA4uJD2YCCiZV2wlXUjtDpuMOgiwIAaSP5Oliui_ZEze7boRWrFnx80nF54YChzBiKWw/s1600/Hadcrut+natural+variability+Summary+2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="493" data-original-width="676" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwPdpTiPKLKRIkmQ5JFjC_7AuxTaivbSPMapuDxFMHRhvRHUmFLusrndT8cRKWBdGbnSb9-Bg_DA4uJD2YCCiZV2wlXUjtDpuMOgiwIAaSP5Oliui_ZEze7boRWrFnx80nF54YChzBiKWw/s640/Hadcrut+natural+variability+Summary+2.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
One other analysis that might be of interest is to gauge how 'The Tidal Model of ENSO' that Paul Pukite has developed fares in this regard. I have one of his model spreadsheets at hand and so I went back to the first analysis using HADCRUT and substituted the model output for the CTI:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj41RIcOq98EZrlur2hro9ikt3fkMvc1DsnPDy7WqwVzCZHVkrJhjZ71ImQG_h3dNAsmVeuygwIyEJqSYpXEF88BOXnTlTWfg9tHmFQUQjexJGXDKKTjS6q22kNiRmk-kSbg6YHLlr7RIoD/s1600/Hadcrut+natural+variability+Summary+3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="493" data-original-width="676" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj41RIcOq98EZrlur2hro9ikt3fkMvc1DsnPDy7WqwVzCZHVkrJhjZ71ImQG_h3dNAsmVeuygwIyEJqSYpXEF88BOXnTlTWfg9tHmFQUQjexJGXDKKTjS6q22kNiRmk-kSbg6YHLlr7RIoD/s640/Hadcrut+natural+variability+Summary+3.png" width="640" /></a></div>
<br />
For this application, the model output works as well as the NINO34 dataset to within 4 ten-thousandths of a degree and actually slightly better than the CTI.Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-90199622041849160842017-09-21T12:15:00.000-07:002017-09-21T12:15:07.230-07:00Reproducing L&W 2005Luckman & Wilson 2005, Fig 1a (red) overlaid with my reproduction (blue)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYGZgaEhw9M7FFu7J2Zn30OwSiXdsDZJV2hBzePSNOHwhOhdwRA5Xep9Voa3FtrBLIAYzbFIco8B_PX7CHxoa39VJVBXNSRu3brK5Cwrk8bzb8WjmTPIlPAWNPpcNOh2w6Uem5_sFmxkoN/s1600/LW+fig1a+comparison.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="400" data-original-width="900" height="284" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYGZgaEhw9M7FFu7J2Zn30OwSiXdsDZJV2hBzePSNOHwhOhdwRA5Xep9Voa3FtrBLIAYzbFIco8B_PX7CHxoa39VJVBXNSRu3brK5Cwrk8bzb8WjmTPIlPAWNPpcNOh2w6Uem5_sFmxkoN/s640/LW+fig1a+comparison.jpg" width="640" /></a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-56574512348448321532017-09-06T14:31:00.001-07:002017-09-06T14:31:07.262-07:00WHUT's Tidal Model of ENSOPaul Pukite (aka WHUT or geoenergymath) has a <a href="http://contextearth.com/2017/04/10/tidal-model-of-enso/">Tidal Model of ENSO</a> based on solar / lunar gravitational forcing. I've been following his travails in working through the his models since he first began writing about <a href="http://contextearth.com/2013/10/26/csalt-model/">CSALT</a> back in 2013.<br />
<br />
WHUT can annoy some people with his constant peddling of his ENSO model, but I really can't blame him. He's found that ENSO is a largely deterministic process that can be both hindcast and forecast with pretty damn good accuracy -- especially when one considers that many believe that ENSO cannot be predicted more than 4 to 6 months in advance and that as a result <span style="background-color: white; color: #333333; font-family: Georgia, "Bitstream Charter", serif; font-size: 16px;">natural variability in climate is chaotic and impossible to predict.</span><br />
<br />
I've seen many dismiss WHUT's claims on general grounds, but never actually dispute any of the math or results. I can only assume they've never actually taken the 10 or 15 minutes time to look at his model and the results. <br />
<br />
He has been treated like a crackpot in some venues and I'm either a fool or they are. Someone needs to tell me what's actually *wrong* with his model as opposed to just dismissing it out-of-hand. A simple graph like this should be enough to catch anyone's eye that's paying any attention whatsoever.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD3Ei4i0vULkqVG2X7j64QJ5K7wDudx3RgOZvCFR0S-QdDGHDucv1XKF-RejEnAIQ4-n1TGN5agIpvSuaz9M9rhClMYyTholLi6I-KM0zMPuWG-7cxfOb7pECBXc3ANtg5PcqfxeRtYT3r/s1600/whut_enso_training.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="239" data-original-width="1134" height="134" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD3Ei4i0vULkqVG2X7j64QJ5K7wDudx3RgOZvCFR0S-QdDGHDucv1XKF-RejEnAIQ4-n1TGN5agIpvSuaz9M9rhClMYyTholLi6I-KM0zMPuWG-7cxfOb7pECBXc3ANtg5PcqfxeRtYT3r/s640/whut_enso_training.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<a href="http://contextearth.com/2017/08/17/enso-model-for-predicting-el-nino-and-la-nina-events/">ENSO model for predicting El Nino and La Nina events</a>Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-19744420053496717882017-08-31T02:26:00.000-07:002017-08-31T02:26:55.169-07:00How do we put tides in GCMs?How do we put tides in GCMs?<br />
<br />
<i>The accuracy of surface elevations in forward global barotropic and baroclinic tide models</i>, Arbic et al, 2004, <a href="https://www.gfdl.noaa.gov/bibliography/related_files/bka0404.pdf">doi:10.1016/j.dsr2.2004.09.014</a> lists 10 different parameters that need to be set. The anomalistic month appears as Mm, but geoenergymath/WHUT's <a href="http://contextearth.com/2017/04/10/tidal-model-of-enso/">Tidal Model of ENSO</a> also requires the tropical and draconic numbers.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_JXHUysj2adyU63wt4eJGzXGRRB0w-Ws5uYmnQGOs749eDi_PXHOs4TwLr3JxYWNTEfVbLeH5xvyPS_9pDrs3H9eEeI-rFFAC5fOdgxlyyfUMKHPRYfvBlbo1n6WjmMSq5zOkPMRutMEp/s1600/arbic_table_1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="285" data-original-width="692" height="163" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_JXHUysj2adyU63wt4eJGzXGRRB0w-Ws5uYmnQGOs749eDi_PXHOs4TwLr3JxYWNTEfVbLeH5xvyPS_9pDrs3H9eEeI-rFFAC5fOdgxlyyfUMKHPRYfvBlbo1n6WjmMSq5zOkPMRutMEp/s400/arbic_table_1.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
The limitations, computational expense, and difficulty in achieving this is probably best spelled out by reading <i>Concurrent simulation of the eddying general circulation and tides in a global ocean model</i>, Arbic et al, 2010, https://doi.org/10.1016/j.ocemod.2010.01.007 <a href="https://hycom.org/attachments/251_tideplusgencirc.pdf">pre-print pdf</a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-78307483314120138562017-08-24T16:14:00.001-07:002017-08-24T16:14:26.580-07:00A 2000 year record of climate variations reconstructed from Haukadalsvatn, West Iceland<div>
Áslaug Geirsdóttir, Gifford H. Miller,Thorvaldur Thordarson, Kristín B. Ólafsdóttir</div>
<div>
Fig 13</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUPeQyhIoxJaI1NHmdq5eYcCKFhOFOZ-PBnN8yc8Bd8bLrzLJIj4XZC3a3JnNbqsoiiDXsazEPsCkSwJOlsIhbT4oz6Obf4UFJQNp9iOGomZuJyJpHcoQPH8PQhrbWm97sL0vqVQPFJw__/s1600/Geirsd%25C3%25B3ttir_2009_Fig13.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="598" data-original-width="610" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUPeQyhIoxJaI1NHmdq5eYcCKFhOFOZ-PBnN8yc8Bd8bLrzLJIj4XZC3a3JnNbqsoiiDXsazEPsCkSwJOlsIhbT4oz6Obf4UFJQNp9iOGomZuJyJpHcoQPH8PQhrbWm97sL0vqVQPFJw__/s1600/Geirsd%25C3%25B3ttir_2009_Fig13.jpg" /></a></div>
<div>
<br /></div>
Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-49368915087761214182017-05-29T13:11:00.001-07:002017-05-30T14:23:27.336-07:00Mores snakes and ExcelIn the previous post I listed the code to find external links in Excel WorkBooks. In practice though I want loop thru and collect every Excel WorkBook in a directory, the Last Modification Date for the file, whether the WorkBook performs a specific function, the total # of and names of all WorkSheets in each WorkBook, and the total # of and names of all Defined Names in the WorkBooks.<br />
<br />
This is what I ended up with:<br />
<br />
<pre style="background-color: white;"><span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small; line-height: 1.1;">#
# excel_info.py
# Find worksheets, defined names and CheckBox1 value in Excel WorkBooks.
# Loop thru directory and write each WorkBook's name, Last Modification Date,
# CheckBox1 value (to see if uncertainties are calculated), Total # of
# WorkSheets, the names of all Worksheets, Total # of Defined Names,and a
# list of all Defined Names into CSV
#
# In the CSV file:
# Column 1 = WorkBook name
# Column 2 = Last Modified Date
# Column 3 = CheckBox1 value
# Column 4 = Total # of WorkSheets in WorkBook
# Column 5 = All Worksheet names
# Column 6 = Total # of Defined Names in WorkBook
# Column 7 = All WorkBook Defined Names
#
</span></pre>
<pre style="background-color: white;"><span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small; line-height: 1.1;">import win32com.client as win32
import os
import glob
import csv
import datetime
from collections import Counter
wb_def_names = Counter()
ws_names = Counter()
os.chdir('C:\origin')
xlfiles = sorted(glob.glob("*.xls"))
outputFile = open('output.csv', 'w', newline='')
outputWriter = csv.writer(outputFile)
xlApp = win32.gencache.EnsureDispatch('Excel.Application')
xlApp.EnableEvents = False
#xlApp.ScreenUpdating = False
cwd = os.getcwd()
this_file = 0
for xlfile in xlfiles:
showunc_val = 0
unccalc_val = bool(False)
showunc = 0
found = 0
checkbox1_caption = "None"
this_file = this_file + 1
if this_file%100 == 0:
print("Processing ",this_file)
if this_file &gt;= 1:
open_ok = 1
try:
xlApp.DisplayAlerts = False
xlwb = xlApp.Workbooks.Open(cwd+"\\"+ xlfile)
except:
open_ok = -1
print("error in open xlfile")
try:
xlwb_name = xlApp.ActiveWorkbook.Name
except:
open_ok = -1
print("error in open ActiveWorkbook.Name")
if open_ok:
cntr = 1
try:
for n in xlApp.ActiveWorkbook.Names:
wb_def_names[(n.Name)] += cntr
found = (str.find(n.Name, "showunc"))
found2 = (str.find(n.Name, "unccalc"))
if found == 0 or found2 == 0:
this_n = str(n)
dummy = this_n.split("!")
dummy2 = str(dummy[0]).replace("=", "")
dummy2 = str(dummy2).replace("'", "")
if found == 0:
dummy3 = str(dummy[1])
showunc_val = xlApp.Sheets(dummy2).Range(dummy3).Value
if found2 == 0:
checkbox1_caption = xlApp.Sheets(dummy2).OLEObjects("CheckBox1").Object.Caption
unccalc_val = xlApp.Sheets(dummy2).OLEObjects("CheckBox1").Object.Value
cntr = cntr + 1
list_wb_def_names = [v for v in wb_def_names.keys()]
except:
open_ok = -1
print("error in ActiveWorkbook.Names",n,n.Name,cntr)
cntr2 = 1
try:
for s in xlApp.ActiveWorkbook.Sheets:
ws_names[(s.Name)] +=cntr2
cntr2 = cntr2 + 1
list_ws_names = [v for v in ws_names.keys()]
except:
open_ok = -1
try:
xlApp.ActiveWorkbook.Saved = True
xlApp.ActiveWorkbook.Close()
except:
open_ok = -1
print("error in closing workbook")
try:
last_modified_date = datetime.datetime.fromtimestamp(os.path.getctime(cwd+"\\"+ xlfile))
except OSError:
last_modified_date = 0
last_modified_date = last_modified_date.strftime("%d-%m-%Y")
try:
outputWriter.writerow([xlwb_name, last_modified_date,unccalc_val,cntr2-1,list_ws_names,cntr-1,list_wb_def_names])
except:
print("error writing to outputfile")
print("Workbook = ", xlwb_name)
wb_def_names.clear()
ws_names.clear()
checkbox1_caption = "None"
else:
print("Unable to process ",xlfile)
else:
dummy5 = 0
#assert isinstance(xlApp.Application, object)
#xlApp.Application.Quit()
outputFile.close()
</span></pre>
<br />
I found that everything worked better if Excel was already open before I ran the program - not sure why. I'm retrieving the value for CheckBox1 and not validating it only because I did when testing the code and using the CheckBox Caption property showed that - in my instance - they were all what I thought they were - thank you Dennis Green :)<br />
<br />
<br />
It doesn't work flawlessly. During runtime I get a quite a few (2 or 3% ??) of the files reporting "error in ActiveWorkbook.Names " from one of my error handlers. It doesn't seem to affect the output, so I haven't bothered to figure out why some files generate the error. I will eventually store just the filenames that create the error and see why.<br />
<br />
A more significant error is that certain poorly written Excel macros in the WorkBook generate errors on opening the WorkBook. As I suspected when writing the previous post, moving the statement<span style="background-color: white; color: #666666; font-family: "courier new" , "courier" , monospace; font-size: xx-small;"> xlApp.DisplayAlerts=False</span> helped a lot. I added <span style="background-color: white; font-family: "courier new" , "courier" , monospace; font-size: xx-small;">xlApp.EnableEvents=False</span> to try and stop macros from causing a problem, but it too may need to be moved inside the main loop to be effective.<br />
<br />
All in all, I know my code isn't up to 'professional' standards, but it works, does what I set out for it to do, and gives me enough knowledge to use it for many other similar projects. Eventually, if I use Python often enough, I'll start building class libraries and user-defined functions, but I'm happy with the results as it is.<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-20073899218559782972017-05-28T12:02:00.002-07:002017-05-28T12:21:24.608-07:00Snakes on a computerWikipedia has a lot to say about Python, in part:<br />
<blockquote class="tr_bq">
<span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;">Since 2003, Python has consistently ranked in the top ten most popular programming languages as measured by the </span><a class="mw-redirect" href="https://en.wikipedia.org/wiki/TIOBE_Programming_Community_Index" style="background: none rgb(255, 255, 255); color: #0b0080; font-family: sans-serif; font-size: 14px; text-decoration-line: none;" title="TIOBE Programming Community Index">TIOBE Programming Community Index</a><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;">. As of March 2017</span><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;">, it is the fifth most popular language.</span><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 11.2px; white-space: nowrap;"> </span><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;">It was ranked as Programming Language of the Year for the year 2007 and 2010.</span><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;"> It is the third most popular language whose </span><a href="https://en.wikipedia.org/wiki/Syntax_(programming_languages)" style="background: none rgb(255, 255, 255); color: #0b0080; font-family: sans-serif; font-size: 14px; text-decoration-line: none;" title="Syntax (programming languages)">grammatical syntax</a><span style="background-color: white; color: #222222; font-family: sans-serif; font-size: 14px;"> is not predominantly based on </span><a href="https://en.wikipedia.org/wiki/C_(programming_language)" style="background: none rgb(255, 255, 255); color: #0b0080; font-family: sans-serif; font-size: 14px; text-decoration-line: none;" title="C (programming language)">C</a>...</blockquote>
<br />
So, why has it taken me so long to play around with it? Yesterday I remedied my ignorance.<br />
<br />
My project was to collect information on 8000 Excel Workbooks. Doing it manually was simply out of the question. In the past I've used GUI macros to automate opening and inspecting 1000s of files, but those were text files. Excel files are - obviously - a different critter altogether. Hence the need for a different approach. Greg Laden has posted some articles regarding Python and those spurred me to investigate.<br />
<br />
Here's the newbie code I wrote to inspect each file for external links:<br />
<br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1: # </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">2: # links.py </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">3: # Find all exteral links referenced in Excel WorkBook </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">4: # Loop thru directory and write each WorkBook's name, and </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">5: # the names of all external links. </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">6: # </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">7: import win32com.client as win32 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">8: import os </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">9: import glob </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">10: import csv </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">11: from collections import Counter </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">12: os.chdir('C:\origin\XLSM') </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">13: xlfiles = sorted(glob.glob("*.xlsm")) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">14: outputFile = open('output.csv', 'w', newline='') </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">15: outputWriter = csv.writer(outputFile) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">16: xlApp = win32.gencache.EnsureDispatch('Excel.Application') </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">17: xlApp.ScreenUpdating = False </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">18: xlApp.DisplayAlerts = False </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">19: cwd = os.getcwd() </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">20: wb_links = [] </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">21: this_file = 0 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">22: for xlfile in xlfiles: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">23: this_file = this_file + 1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">24: if (this_file >= 1): # Mostly for debug purposes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">25: open_ok = 1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">26: try: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">27: xlwb = xlApp.Workbooks.Open(cwd+"\\"+ xlfile) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">28: except: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">29: open_ok = -1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">30: try: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">31: xlwb_name = xlApp.ActiveWorkbook.Name </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">32: except: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">33: open_ok = -1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">34: if open_ok: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">35: try: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">36: wb_links = xlApp.ActiveWorkbook.LinkSources() </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">37: print("Workbook = ", xlwb_name, wb_links) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">38: except: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">39: open_ok = -1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">40: print("Unable to process ",xlfile) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">41: try: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">42: xlApp.ActiveWorkbook.Saved = True </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">43: except: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">44: open_ok = -1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">45: try: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">46: xlApp.ActiveWorkbook.Close() </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">47: except: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">48: open_ok = -1 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">49: outputWriter.writerow([xlwb_name, wb_links]) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">50: else: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">51: print("Unable to process ",xlfile) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">52: else: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">53: dummy = 0 </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">54: assert isinstance(xlApp.Application, object) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">55: xlApp.Application.Quit() </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">56: outputFile.close() </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br />
</span></div>
<div>
While this code just collects the external links, in the end my program collected the filename of each Workbook, file size, last modification date, names of all Worksheets, names and ranges of all Defined Names, external links and the cell value for a specific Defined Name of interest and wrote it all to a CSV file.<br />
<br />
The only real problem is <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">18: xlApp.DisplayAlerts = False </span>When the program is first run this seems to be working fine, but inevitably there is a file that causes an error in Excel that isn't covered by the 'DisplayAlerts" function. When that happens it seems to turn 'DisplayAlerts' back on. <br />
<br />
I might try to move Line 18 into the main loop to see if that works better, but otherwise I collected all the information I was interested in. Total runtime (from within PyCharm) was close to two hours.<br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
</div>
</div>
Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-52899978505008335072017-05-20T07:26:00.001-07:002017-05-20T07:39:30.243-07:00CluelessI think the woman on the left may be grandma - and possibly aunt Kelly in the middle with the glasses. But that's just a WAG. It could be grandma with her brothers and sisters - but I don't know how many she actually had.<br />
<br />
The men in the 2nd photo must be some relation on mom's side, but not the faintest idea who they are.<br />
<br />
These were scanned in from old negatives, palate inverted, then brightness and contrast adjusted to get something viewable.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG19zmJRCISWUfcpLUCZs5i9rsFoFi2pInSi-cWr7C-wWc13szy58zogbfrI_9s_DtnDIHcNXcGfiBngkIuBWOQw9v33NYvGicbmbUQbjcBr8HF07TXfnSFmfdaWaULIWyGXMW1Wt8vkvB/s1600/grandma+-+maybe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG19zmJRCISWUfcpLUCZs5i9rsFoFi2pInSi-cWr7C-wWc13szy58zogbfrI_9s_DtnDIHcNXcGfiBngkIuBWOQw9v33NYvGicbmbUQbjcBr8HF07TXfnSFmfdaWaULIWyGXMW1Wt8vkvB/s640/grandma+-+maybe.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh9xuv80KHCoiwyvbP4XYB-QaSFLtiLbMqtzHJtPEdVLUaYh7-dIcYN5v9gP3RaJp6lqMWVE4Q5zi0_Qj_GJPOuCPrAzJeZu_v-NFfx9ycnrmXrwN8Deu1CNu6pXcCyoiMq997WmaYhkP4/s1600/not+a+clue+who.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh9xuv80KHCoiwyvbP4XYB-QaSFLtiLbMqtzHJtPEdVLUaYh7-dIcYN5v9gP3RaJp6lqMWVE4Q5zi0_Qj_GJPOuCPrAzJeZu_v-NFfx9ycnrmXrwN8Deu1CNu6pXcCyoiMq997WmaYhkP4/s640/not+a+clue+who.png" width="550" /></a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-15143903162689285542017-05-20T06:45:00.002-07:002017-05-20T06:45:31.186-07:00More old photosImpossible to say if that's Blanche or Tudy with (I assume) their dad - George Hammond.<br />
<br />
Love the glasses on that state ID photo. Surprised it took her 4 months after her 18th birthday to get an ID.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizuA0QYZFD7n-4X0ejT3M7wF1SsizkXR-QcATbMafeGTR-5tA6_gMXxvlg15m9M2IqCc_b17iQvrIsBWh0owZiDXwUvTlJPF_N2CO1YipY5uj_kzqMkjNJHoQKxMCtNkexbKvjn2AM30BO/s1600/Mom+with+her+dad+and+photo+ID.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizuA0QYZFD7n-4X0ejT3M7wF1SsizkXR-QcATbMafeGTR-5tA6_gMXxvlg15m9M2IqCc_b17iQvrIsBWh0owZiDXwUvTlJPF_N2CO1YipY5uj_kzqMkjNJHoQKxMCtNkexbKvjn2AM30BO/s640/Mom+with+her+dad+and+photo+ID.png" width="548" /></a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-89088143632532825242017-05-18T17:00:00.003-07:002017-05-18T17:00:59.336-07:00Old photosHer birthday and Mother's Day often fell on the same day.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaA-_UdxP9vIUb-SV83ar51wyPqXLjPl2GGtKT70Zawd_HSqwcmQN7xlFR4dbcFlmedpnC7MWvkk3pYoekURhSriikLRmMGi9cR7c0JY0Ftrbw8aNr3fOz_R69Ja_xWMMbPbH5pnHmf5Lc/s1600/Blanche+Burch+1957.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaA-_UdxP9vIUb-SV83ar51wyPqXLjPl2GGtKT70Zawd_HSqwcmQN7xlFR4dbcFlmedpnC7MWvkk3pYoekURhSriikLRmMGi9cR7c0JY0Ftrbw8aNr3fOz_R69Ja_xWMMbPbH5pnHmf5Lc/s640/Blanche+Burch+1957.jpg" width="501" /></a></div>
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-89813335376532534102017-04-20T10:40:00.000-07:002017-04-20T10:40:30.956-07:00<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTf0VkYTbhuM7-E5BlAdu95aErUxgoeRNqY3l4hjbKHP8CIJHVMlpB5DP4jjpigNWTS1gXNH3wiM4UhEL4sKCodR2rb9ZC35LPX6Xo8FJfXqh3NEFxxqGLp5-WQ7giF1smS9JBFv2jIeal/s1600/emiss+%252B+trueamo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTf0VkYTbhuM7-E5BlAdu95aErUxgoeRNqY3l4hjbKHP8CIJHVMlpB5DP4jjpigNWTS1gXNH3wiM4UhEL4sKCodR2rb9ZC35LPX6Xo8FJfXqh3NEFxxqGLp5-WQ7giF1smS9JBFv2jIeal/s320/emiss+%252B+trueamo.png" width="320" /></a></div>
76, <b style="background-color: white; color: #252525; font-family: sans-serif; font-size: 14px;">Habibullo Ismailovich Abdussamatov</b><br />
<b style="background-color: white; color: #252525; font-family: sans-serif; font-size: 14px;"><br /></b>Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-66179281935783298602017-02-16T08:45:00.001-08:002017-02-16T08:45:03.087-08:00FDDs DMI N80 thru Feb 15<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJjDEWyHmsk9Tlnf2HyFxENftrPeHukdzC7YJwkzLXs_8buDnVwT1qeswmsz7UAA9ffIyW4bJ_la_74M0ANvfQlMwSdvARt0AKb9fiTSJ9VscFFa6qHNTRjtwNePsM0NAiI3YxVhSh-BxO/s1600/Anomaly+FDDs_2_15_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJjDEWyHmsk9Tlnf2HyFxENftrPeHukdzC7YJwkzLXs_8buDnVwT1qeswmsz7UAA9ffIyW4bJ_la_74M0ANvfQlMwSdvARt0AKb9fiTSJ9VscFFa6qHNTRjtwNePsM0NAiI3YxVhSh-BxO/s640/Anomaly+FDDs_2_15_2017.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ2V_7LkTfypZ78k5igeelOpJvGmwD5rFjJ9Wjf7Eg_5ZLVYKrd0A9kER7PAy0LtPg9R9ZiJSf0YosmFLQdhOwfsSdmpux23pomyZYHv53u3nFE8zcNP0nUqkr_l_eeRd1ezn8aO58Pf4m/s1600/Total+FDDs_2_15_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ2V_7LkTfypZ78k5igeelOpJvGmwD5rFjJ9Wjf7Eg_5ZLVYKrd0A9kER7PAy0LtPg9R9ZiJSf0YosmFLQdhOwfsSdmpux23pomyZYHv53u3nFE8zcNP0nUqkr_l_eeRd1ezn8aO58Pf4m/s640/Total+FDDs_2_15_2017.png" width="640" /></a></div>
<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-85558270695292616052017-02-09T15:15:00.005-08:002017-02-09T15:15:55.274-08:00FDDs DMI N80 thru Feb 8<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj57X0HPpF6R7m_7FJOInbMXFJE7F9M39YbVp32IxpYgOnfrbievkrw48x9nrCL233th3tuMatjgdKIRD3aBFy4QnQXZvFa66Ak0fnMJy5OkEyL7Z5Gz8L426KXwTcJsQB34DNSYN8BG1oI/s1600/Total+FDDs_2_8_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj57X0HPpF6R7m_7FJOInbMXFJE7F9M39YbVp32IxpYgOnfrbievkrw48x9nrCL233th3tuMatjgdKIRD3aBFy4QnQXZvFa66Ak0fnMJy5OkEyL7Z5Gz8L426KXwTcJsQB34DNSYN8BG1oI/s640/Total+FDDs_2_8_2017.png" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeZTnO0IAUmiRhp12XXxu8MxBJEisQPMZLhpem08dVPjzokX-btuAjZ9KLVYFOkG_cfVwxwKgtUMWzpQhASJ4nhJN5jbGWDHVchFFQ_2L9MtcaLAZglSEU_HUWTK5qkhYTRn1WU52lVzjJ/s1600/Anomaly+FDDs_2_8_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeZTnO0IAUmiRhp12XXxu8MxBJEisQPMZLhpem08dVPjzokX-btuAjZ9KLVYFOkG_cfVwxwKgtUMWzpQhASJ4nhJN5jbGWDHVchFFQ_2L9MtcaLAZglSEU_HUWTK5qkhYTRn1WU52lVzjJ/s640/Anomaly+FDDs_2_8_2017.png" width="640" /></a></div>
<br />
<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-61337948317291575672017-02-08T09:01:00.000-08:002017-02-08T09:01:10.390-08:00Means and mean errorThree sets of synthetic data:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPkwxDH2BV9PaWkbmBioU-N-ukvBMwNCvxz1N6yRjteqsSV3hRftFS5_7ZDHVjYrPvTnrgA1x54iWkbIVKuOQoxrTArzSPyIlC4XdUM7qc51AbDsz1mK8139e9i2aLA1geX8OK0KDNgs87/s1600/Means.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPkwxDH2BV9PaWkbmBioU-N-ukvBMwNCvxz1N6yRjteqsSV3hRftFS5_7ZDHVjYrPvTnrgA1x54iWkbIVKuOQoxrTArzSPyIlC4XdUM7qc51AbDsz1mK8139e9i2aLA1geX8OK0KDNgs87/s640/Means.png" width="640" /></a></div>
Set L is a linear function increasing in value with each year. Set R is a random normal distribution, and Set S is a sine function. Do they RMSEs in and of themselves tell us anything about these functions? No. Only by looking at the actual data would we be able to see this.<br />
<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-37972082220455299992017-02-07T07:09:00.002-08:002017-02-07T07:09:35.647-08:00Someone forgot to tell the arctic to wait for Karl to bust the pause.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_AZjXO8aIh5dY97FLPkLpyEN9ejmX8OtIbt-nvtm6-pQO8tQEf79tRpXnF7Do9vD-qqO84R22kJFpGmFP-9Nb9onyYeNEJJkeFQL9UGZIhGfaPsqWANWUuIOVWMoNpW7LCyTbpZITsIdD/s1600/ASIV+1_31_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="474" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_AZjXO8aIh5dY97FLPkLpyEN9ejmX8OtIbt-nvtm6-pQO8tQEf79tRpXnF7Do9vD-qqO84R22kJFpGmFP-9Nb9onyYeNEJJkeFQL9UGZIhGfaPsqWANWUuIOVWMoNpW7LCyTbpZITsIdD/s640/ASIV+1_31_2017.png" width="640" /></a></div>
<br />
<br />
I was going to add text and arrow for "Curry jumps shark" but there would have been too many arrows and the graph too cluttered.Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-58739627231215413422017-02-04T09:51:00.001-08:002017-02-04T09:51:53.816-08:00Annual FDDs per DMI N80Annual Total Accumulated Freezing Degree Days<br />
(August 1st thru May 31st)<br />
<br />
Does not include this freezing season as it is not yet complete.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMXkRX6jZc7jfs36h1uoLRPFFP-oj1LFsbIwhwYTzAdvX6fdGr83AXkNNpkxr99vL7e9p9pcllljZ4NQgtXoxdbNLCfpe4zShZhZW9-DH7r0M4MzOmszBEIV-MvP4LP3vW_b4z4jp7fhog/s1600/Total+FDDs+N80.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="442" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMXkRX6jZc7jfs36h1uoLRPFFP-oj1LFsbIwhwYTzAdvX6fdGr83AXkNNpkxr99vL7e9p9pcllljZ4NQgtXoxdbNLCfpe4zShZhZW9-DH7r0M4MzOmszBEIV-MvP4LP3vW_b4z4jp7fhog/s640/Total+FDDs+N80.png" width="640" /></a></div>
So, what's happening during the summer months of June & July?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjks55Tn00ReyIEZIL7RvWiaR0IklsCbQPj3PldcOcK1-t0VkqpsM4fvhKog-YxZgUQkc4F6tam_zfybAnkgIIGam2jJrcb-5yUfmmjeAjDV_ELksdV8WfhZ8J2g4ikRJ-jL5KunB70kKv6/s1600/DMI+N80+June+July+Temps.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="408" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjks55Tn00ReyIEZIL7RvWiaR0IklsCbQPj3PldcOcK1-t0VkqpsM4fvhKog-YxZgUQkc4F6tam_zfybAnkgIIGam2jJrcb-5yUfmmjeAjDV_ELksdV8WfhZ8J2g4ikRJ-jL5KunB70kKv6/s640/DMI+N80+June+July+Temps.png" width="640" /></a></div>
<br />
Odd, melt season temperatures are actually *decreasing* - now why would that be?<br />
<br />
<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-27661103412760024232017-02-02T17:29:00.001-08:002017-02-02T17:29:24.942-08:00N80 Freezing Degree Days thru Feb 2, 2017Freezing Degree Days thru Feb 2, 2017<br />
(Climatology = 1958 to 2002)<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
Accumulated FDDs </div>
<div class="separator" style="clear: both;">
Climatology: 3095.9</div>
<div class="separator" style="clear: both;">
2016-7: 1833.0</div>
<div class="separator" style="clear: both;">
Anomaly: -1262.9</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxnvWAd5aAUH2O9h18osQ_Gc6bNmzuG7qiKiq1_pfrrJ4atJENxOh51G6bGXDOs9hUj5s0sFhf72Fq9rRxdU3HfrQEvn1NbxAFm77g-MEoKaT6n-YjixM37OewhiR5TUJ-MFyre6xvMh5_/s1600/Total+FDDs_2_2_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxnvWAd5aAUH2O9h18osQ_Gc6bNmzuG7qiKiq1_pfrrJ4atJENxOh51G6bGXDOs9hUj5s0sFhf72Fq9rRxdU3HfrQEvn1NbxAFm77g-MEoKaT6n-YjixM37OewhiR5TUJ-MFyre6xvMh5_/s640/Total+FDDs_2_2_2017.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKn6PQtnp1sONenyskjqW3KNt-n5KsoB4VKqJZ30JXps0tOXin2H_pQdZ7Vcuap3_k3Jhxtsde8YaeuhefDI9mNFnJo92w6oCu8hIVtQSmg3dXlHKmbjk2IDaaxXT4PTGaGn7UAXf65Gc4/s1600/FDDs_2_2_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKn6PQtnp1sONenyskjqW3KNt-n5KsoB4VKqJZ30JXps0tOXin2H_pQdZ7Vcuap3_k3Jhxtsde8YaeuhefDI9mNFnJo92w6oCu8hIVtQSmg3dXlHKmbjk2IDaaxXT4PTGaGn7UAXf65Gc4/s640/FDDs_2_2_2017.png" width="640" /></a></div>
Implied new ice thickness to date:<br />
Per Lebedev:<br />
Climo: 1.755 m<br />
2016-7: 1.288 m <br />
<br />
Per Bilello:<br />
Climo: 1.408 m<br />
2016-7: 1.039 m<br />
<div class="separator" style="clear: both;">
<br /></div>
Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-64584007293963692742017-01-26T19:59:00.000-08:002017-01-26T19:59:02.475-08:00Accumulated Freezing Degree Days N80 thru Jan 26, 2017Accumulated Freezing Degree Days N80 thru Jan 26, 2017:<br />
Climatology: 2899.4<br />
2016: 1683.9<br />
Anomaly: -1215.4<br />
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5672eje2NL1wcqc5xL9BYgB0iXPUiKDDVYHqF81Ai-TchSAuSk6Ks7Qu82gxVoOKQc8q660r2pCat05HMrtY0ldYZKwMHJ_S7X74Xy9v0JUhqU8gJZrBJ9UIt3ScRkbJUPjlH8IHsRsg1/s1600/Total+FDDs_1_26_2017.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="387" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5672eje2NL1wcqc5xL9BYgB0iXPUiKDDVYHqF81Ai-TchSAuSk6Ks7Qu82gxVoOKQc8q660r2pCat05HMrtY0ldYZKwMHJ_S7X74Xy9v0JUhqU8gJZrBJ9UIt3ScRkbJUPjlH8IHsRsg1/s640/Total+FDDs_1_26_2017.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Implied new ice thickness to date:<br />
Per Lebedev:<br />
Climo: 1.688 m<br />
2016: 1.225 m<br />
<br />
Per Billelo<br />
Climo: 1.355 m<br />
2016: 0.989 m</div>
<div>
<br /></div>
</div>
<div>
<br /></div>
<div>
Note: Anyone claiming a "recovery" in extent is ignoring new ice thickness this year is 27% less than climatology and 10 to 15% less than the average of the past decade. The accumulated FDD anomaly is now greater than at *any* time in the DMI N80 dataset with 40% of the freezing season still to go.</div>
<div>
<br /></div>
Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-54501227824345491952016-12-29T14:10:00.003-08:002016-12-29T14:10:29.396-08:00Accumulated Freezing Degree Days N80 thru Dec 29, 2016Accumulated Freezing Degree Days N80 thru Dec 29, 2016:<br />
Climatology: 2141.1<br />
2016: 1138.6<br />
Anomaly: -1002.5<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY9Kc3dpA2fOpNmYyoPZpYy5hRN-H1ToKvGhyLMM3k3l59f1NQ-IoyPDor6ENs0KBRrxqK5yYYoEe0dlhJCkIhtTsvgjloY_BemQobAfYi5MkV1G4waqFS1ssCF2CSkpdjM60JUKm3gH7j/s1600/Total+FDDs_12_29_2016.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY9Kc3dpA2fOpNmYyoPZpYy5hRN-H1ToKvGhyLMM3k3l59f1NQ-IoyPDor6ENs0KBRrxqK5yYYoEe0dlhJCkIhtTsvgjloY_BemQobAfYi5MkV1G4waqFS1ssCF2CSkpdjM60JUKm3gH7j/s1600/Total+FDDs_12_29_2016.png" /></a></div>
Implied new ice thickness to date:<br />
Per Lebedev:<br />
Climo: 1.412 m<br />
2016: 0.973 m<br />
<br />
Per Billelo<br />
Climo: 1.137 m<br />
2016: 0.788 m<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-55972323555734610742016-12-24T06:50:00.002-08:002016-12-24T06:52:30.366-08:00Record North Atlantic significant wave height2013's record Atlantic significant (17 1/2 minute average) wave height. The record significant wave height was reported as 19m, but the maximum wave height was 27 m.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4vVw1skS8A1NuGUPNVSZCuTeNXpgGvudzFasc5mVMDVsYh7HacrqBRloHi-CRUR3Czi8_FpB38rkra4GkPAstqy7dSjAFWnx2sGugNhyphenhyphennpWE0gJ2ipBzIj037tefvx0mPPgDfQ95OXMtx/s1600/atlantic+wave+2013.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4vVw1skS8A1NuGUPNVSZCuTeNXpgGvudzFasc5mVMDVsYh7HacrqBRloHi-CRUR3Czi8_FpB38rkra4GkPAstqy7dSjAFWnx2sGugNhyphenhyphennpWE0gJ2ipBzIj037tefvx0mPPgDfQ95OXMtx/s640/atlantic+wave+2013.png" width="640" /></a></div>
<br />
Graph from ASU's WMO data archive.Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-30498349042462967892016-12-17T11:29:00.001-08:002016-12-17T11:29:59.605-08:00Accumulated Freezing Degree Days thru Dec 16, 2016Accumulated Freezing Degree Days N80 thru Dec 16, 2016:<br />
Climatology:1787.3<br />
2016: 944.3<br />
Anomaly: -843<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-IRa4Xczuso2slVMkUojxes61m8Bztp5tSTPLu4JaoyhpmxUCWAYIdoTQJG1WmyfBlXvguasc6Zq1yaM_vp6VM9yerifNRmur-ExZuAx5AM4RH-kMnD3No7vV0unx3eDyTJYGOwks85/s1600/Total+FDDs_12_16_2016.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOU-IRa4Xczuso2slVMkUojxes61m8Bztp5tSTPLu4JaoyhpmxUCWAYIdoTQJG1WmyfBlXvguasc6Zq1yaM_vp6VM9yerifNRmur-ExZuAx5AM4RH-kMnD3No7vV0unx3eDyTJYGOwks85/s640/Total+FDDs_12_16_2016.png" width="640" /></a></div>
<br />
<br />
Implied new ice thickness to date:<br />
Per Lebedev:<br />
Climo: 1.269 m<br />
2016: 0.871 m<br />
<br />
Per Billelo<br />
Climo: 1.024 m<br />
2016: 0.707 m<br />
<div>
<br /></div>
Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0tag:blogger.com,1999:blog-4788672781731855663.post-3890489514523477762016-12-11T18:03:00.000-08:002016-12-11T18:10:23.557-08:00Great-great-grandparentsWilliam (b. 1823) & Mary O'Neill (b.1825) arrived in the U.S. prior to 1846. Both were born in Ireland.<br />
Children: Bridget (b.1846, NY)<br />
Ann (b.1847, Maine)<br />
Joseph (b.1853, NY)<br />
Mary (b.1855, NY)<br />
Matty (b.1858, WI)<br />
Records: 1860 Federal Census, Town of Lake, County of Milwaukee, New Coelm P.O.<br />
<br />
Joseph O'Neill married Margaret McDonough (b.1825, NY) most likely in or near Muskego, WI after 1870.<br />
Children: Mary (b.1875)<br />
William (b.1877)<br />
Edward (b. 1879)<br />
Thomas (b.1880)<br />
Joseph (b.1882)<br />
George (b.Dec,6,1883)<br />
Katherine (b.1889)<br />
Walter (b.1892)<br />
<br />
<br />
Grandfather - George (b.Dec,6,1883)<br />
Grandmother - Alice Beaulieu<br />
Great-grandfather - Joseph (b.1853, NY) died 1895 (Chippewa County)<br />
Great-grandmother - Margaret (nee McDonough b.1825, NY) died 1916 (Superior)<br />
Great-great-grandfather - William (b. 1823)<br />
Great-great-grandmother - Mary (nee ??? b.1825)<br />
<br />
<br />Kevin O'Neillhttp://www.blogger.com/profile/06692943768484857724noreply@blogger.com0