iolite Python for LA-ICP-MS Course - Week 4 Discussion

Joe

Hi all,

The week 4 content has now been posted! This week I show you how to use Qt to create simple user interfaces and matplotlib to create plots. I then show you how these two packages can be used to create a QA/QC module for iolite including settings and plots.

This week's assignment is to write a QA/QC module of your own. Since many of you likely already have data you can work with on this we've decide to give you some options. The first is to write a QA/QC module that checks the uncertainty of individual selections in a group. The second is similar but instead looks at LODs. See the video and course notes for more information. Of course, you can also come up with your own idea for a QA/QC module, but please contact us first to make sure you're not taking on too big of a project.

If you have any questions at all about the content this week, please just let us know below.

Happy Learning!

Nathan

Hi,
when I run my QAQC module I get an error message in the python console
I can escape this error message using a try/except but I don't understand why the default settings are not taken into account. Despite the message there is default settings when I open the setting box.
All the best
Nathan

Joe

Nathan

It looks like you're using group.selections() before doing something like group = data.selectionGroup(settings['GroupName']). If you post your code (or the relevant part of it) I should be able to give a more specific fix.

All the best,

Nathan

As it stands now

uncertainty-check.py

4kB

Cheers

Nathan

Joe

Nathan

I was able to get it working by changing the Channels default setting. As it was, it was setting it to just a string by default, so when you try to do a list comprehension on the string (line 35) it takes each letter at a time rather than each channel name. I changed it to:

qaqc.setDefaultSetting('Channels', data.timeSeriesNames(data.Output)[0:1])

By specifying the range 0:1, you still get the first name, but it'll keep it as a list.

In case it hasn't come up yet, when using try/except, it can be helpful to do something like:

try:
    some_code_that_might_raise_an_exception()
except Exception as e:
    print(f'There was an error: {e}')

I probably should have done this in my example. That way, you can handle the error, but also see it. This can be useful because it is tempting to wrap lots of stuff in try/except, but you may be catching and ignoring errors that you didn't anticipate, which will make your code tricky to debug.

Great job on the QA/QC module by the way! With that little fix, it appeared to be working and seemed useful when looking at our DRO4 dataset.

All the best,

Nathan

Thanks Joe,
I still get the same error message if I take the problematic line (definition of the df) out of the try/except after changing the default setting as you suggested.. But given what you said I will try to investigate myself. It doesn't prevent the module to work anyway :-)
Thanks for the tip about rising the error as a print

All the best

Nathan

Hi Joe,
I think the problem is actually that the update fcn does not read the settings as long as I do not open the setting widget. So when I open the module it has this error and do not print any of the default settings as it should with l.29. Once I open the settings widget it does work properly. In the attached code I cheated in the try/except by putting the default setting in the except part (l.36). But I do not see how to 'force' the module to read the default settings when I open it.

uncertainty-check.py

5kB

All the best

Nathan

Joe

Nathan

Good point! You could try putting the default settings outside of any function. Before either function is called, the whole file is evaluated, so I think that should initialize the default settings no matter what.

All the best,

Nathan

Thanks Joe,
All the best

rosieluain

I'm attempting to write a module for the LOD option. There are 3 methods for getting detection limits. Instead of having the user select which detection limits they used in the DRS, I was hoping to get it automatically through the module. I am thinking of pulling the data for each method, and then removing the columns where the values are "None". Is there an easier way to do this, like calling it from a saved setting somewhere?

This week is definitely the most challenging for me so far, but learning lots! Thanks!

Joe

rosieluain

Recent versions of iolite will actually calculate all three versions all the time. For the original "Trace Elements" DRS, at least one of the LOD checkboxes needs to be checked, but nothing needs to be done for "Trace Elements Next" or "3D Trace Elements". For the Pettke LOD calculation to work, you'll need dwell times set. And for the Longerich LOD to work, you'll need backgrounds > 0.

So, you could make an option to select which version to use in the QA/QC module settings.

To answer your question, you can drop columns where the values are not a number using "dropna". In your case you might do something like:

df = df.dropna(axis='columns', how='all')

Documentation for dropna is here.

All the best,

rosieluain

Joe Thanks Joe! That was really helpful in figuring out what the data might be (un)available and what errors to look for and make messages for.

kai2

This weeks assignment definitively separates the noobs from the pros! I tried to follow your introduction into the whole gui thing with widgets and matplotlib but i have to admit that I understood close to nothing 😃 If I just go ahead and copypaste your code I might get it to work but the feeling remains that i walked over a bridge that wasn't there. I'll bail out for now and will try to catch up at my own pace later (after I've read a few more books on python programming). In any case, this is an excellent source of information and I can see how this could be super helpful for my work, so many thanks for doing this course!!

Cheers, kai

mmcgregor

is there a template for this?

Joe

mmcgregor

You can use the built in one by clicking the "New" button in the QA/QC view.

All the best,

aredakau

Hi Joe,

I am behind the course but trying to catch up. I am struggling with the same error that Nathan had but could not solve it.

I tried this tip:

try:
some_code_that_might_raise_an_exception()
except Exception as e:
print(f'There was an error: {e}')

and got the following error:

There was an error: std::runtime_error: [DataManager::timeSeries] no matches for N
Traceback (most recent call last):
File "C:/iolite4 data/Plugins/QA-QC/Uncertanity Check.py", line 57, in update
df = data.frame(group.selections(), ['Name'], channels, ['mean', 'int2se'])
UnboundLocalError: local variable 'channels' referenced before assignment

This error appears with

try:
    channels = [data.timeSeries(c) for c in settings['Channels']]

Could you please check the code for me?
Kind regards,
Ahmad

uncertanity-check.py

3kB

Joe

aredakau

I think the problem is that your settings['Channels'] is not a list of strings but just a string (perhaps starting with Na?) So when you do the list comprehension to create channels it is iterating the characters of a string rather than the strings of a list.

And I think you're seeing that because the default setting is set to a single channel name data.timeSeriesNames()[0] rather than a list. You could try just wrap that in square brackets, e.g.

qaqc.setDefaultSetting('Channels', [data.timeSeriesNames()[0]])

All the best,

tonydoss

Hi team! I think I'm having the same problem as Nathan too. I followed Joe's tutorial, trying to do the same, and get the following error:

df = data.frame(group.selections(), ['Name'], channels, ['mean', 'int2se']) UnboundLocalError: local variable 'group' referenced before assignment

then if I click on the module Settings box, I get:

df = data.frame(group.selections(), ['Name'], channels, ['mean', 'int2se'])
UnboundLocalError: local variable 'channels' referenced before assignment

I've loaded the Gabbro data set so there are channels. I may have missed the instructions about what to do with this data set, but though I'd take matters into my own hands and defined selections for the standards, samples and baseline 🙂
despite all this, I get the errors above.
Any ideas why the script can't find 'group' and 'channels'? (script attached)

test.py

3kB

Related question: is there a tutorial about Iolite python objects I've missed? There is a huge gap - in my head - between the python tutorials of weeks 1 and 2, and getting straight into using iolite objects in week 3. For instance, in week 3 (and 4) I followed the tutorial and wrote code, but don't understand 1/2 of what I write (maybe that's how Kai feels too haha)
For instance, why do we write from datetime import datetime instead of import datetime (OK, that's probably pure python there). Or what is importer.fileName.endswith? I guess importer is a new object we define in the script, but fileName is an Iolite object, and endswith is a python object? I looked into the documentation without much success unfortunately.

Joe

tonydoss

Regarding the UnboundLocalError, my guess is that the try/except before that data.frame(...) has an exception and therefore group is not defined. Since you can't get the DataFrame without a group setting, I suggest changing the pass to return, e.g.,

    try:
        group = data.selectionGroup(settings['GroupName'])
        channels = [data.timeSeries(c) for c in settings['Channels']]
        max_rsd = settings['MaxRSD']
        zscore_cutoff = settings['ZScoreCutoff']
    except:
        qaqc.pushHtml('Cannot run with these settings')
        return # !! Change this from pass to return

That will get rid of the error you're seeing, but there may still be an issue with your settings widget if you are unable to configure the group and have it update properly. Please let me know what happens with the above change.

Regarding iolite python objects, we have been trying to introduce the new iolite-specific objects we use each week. For example, Bence briefly mentions the importer object at ~ 26:30 into the importers lesson and then uses it in the example. If one of the iolite-specific objects or types needs more explanation, we're happy to provide that as needed. Finding the right level of detail for the videos can be tricky. Also, as you point out, there is a mix of iolite-specific and non-iolite stuff, and for people that are new to python knowing which is which can be difficult!

The confusing thing about datetime is that it is the name of the python module for dealing with dates and times, but that module defines a datetime type that represents a date + time in one object. See here where you can see that the datetime module defines several types, one of which is datetime. So to get the datetime type from the datetime module we do from datetime import datetime.

importer is an object that iolite inserts into your import script before it is used. This importer object allows you to get the file name being passed to the importer as importer.fileName (among other things), and importer.fileName is just a python string. Since it is a python string, you can call endswith(...) on it to check if it ends with the string provided as an argument. The documentation for python strings can be found here.

All the best,

tonydoss

thanks Joe! I changed pass to return; when I run the module and the window appears, it still says "Cannot run with these settings":

I then click on Settings. For some reasons, it shows Baseline as the default group and 10 as default MaxRSD, but nothing for default channel and ZScoreCutoff, even though I wrote in the script (with proper indentations, which doesn't show below):
def settingsWidget(): try: qaqc.setDefaultSetting("GroupName", data.selectionGroupNames()[0]) qaqc.setDefaultSetting("ChannelName", data.timeSeriesNames()[0]) qaqc.setDefaultSetting("MaxRSD", 10) qaqc.setDefaultSetting("ZScoreCutoff", 2) except: pass

I select Group G_BCR2G, and when I select Channel Input>Na23, the console displays this:
File "C:/Users/tonyd/Documents/iolite/Plugins/QA-QC/test.py", line 52, in update img = QImage(bytes(plt.gcf().canvas.buffer_rgba()), w, h, QImage.Format_RGBA8888) File "C:\Program Files (x86)/iolite-software/iolite4/python3.8/site-packages\matplotlib\backends/backend_agg.py", line 448, in buffer_rgba return self.renderer.buffer_rgba() AttributeError: 'FigureCanvasAgg' object has no attribute 'renderer'

Not sure what it doesn't like with img = QImage(bytes(plt.gcf().canvas.buffer_rgba()), w, h, QImage.Format_RGBA8888)

If I try clicking on Settings again, the widget doesn't appear and I get again: File "C:/Users/tonyd/Documents/iolite/Plugins/QA-QC/test.py", line 52, in update img = QImage(bytes(plt.gcf().canvas.buffer_rgba()), w, h, QImage.Format_RGBA8888) File "C:\Program Files (x86)/iolite-software/iolite4/python3.8/site-packages\matplotlib\backends/backend_agg.py", line 448, in buffer_rgba return self.renderer.buffer_rgba() AttributeError: 'FigureCanvasAgg' object has no attribute 'renderer'

I closed the window, re-ran the module and tried different combinations of channel and group; same error.

Joe

tonydoss

Can you post the current version of your script? The one I'm looking at that you sent last time has an issue, but I'm not sure it is your current issue. Specifically, in update() you're doing

channels = [data.timeSeries(c) for c in settings['Channels']]

but in the settingsWidget() you're doing

qaqc.setDefaultSetting("ChannelName", data.timeSeriesNames()[0])

So, the name being used for the channels setting is different (i.e. "ChannelName" vs "Channels"), and in the settings you're setting it to a single channel, not a list of channels, but the update() is expecting a list of channels.

There also appears to be a setting name mismatch in the part at the end of settingsWidget() where you set the current value of the control based on the stored setting.

The issue with the figure might be that you're calling plt.gcf().canvas.draw rather than plt.gcf().canvas.draw() (i.e. missing the brackets on the end). I can see how this is a bit confusing, e.g., why do gcf() and draw() have brackets while canvas does not? This comes down to gcf being a function of the plt object, canvas being an attribute of the figure object returned from gcf(), and draw() being a function of the canvas. If that just makes things more confusing, forget I mentioned it!

Maybe these issues have already been sorted out?