PCA map loading?

JakubS

Hi all,

I'm super excited about the new PCA feature, and have started trying it out.

After poking around for some time, I've not been able to find any the PCA metadata (i.e. the coefficients, % explained etc). I've been able to generate the PC maps, but under the Channels tab, there is no information about what goes into each PC.

Thanks in advance for any help!

Jakub

Joe

Bence

JakubS

The coefficients actually are reported in the messages, but not % explained.

All the best,

Bence

Hi Jakub,

Sorry, those metadata are not available currently, but it is a great idea. We'll look into including them.

I'll post here when we have an update.

-Bence

JakubS

Yes indeed! Apologies, I missed this!

As a followup question, the results I get are quite different from what I would expect (after some post-processing in MATLAB). Are the channels z-score normalized before PCA?

Thanks in advance!

Jakub

Joe

JakubS

No, they were not. I've added some normalization options now and that'll be in the next release.

I've also added the explained variance to the messages output and as channel properties on the PCA channels.

All the best,

JakubS

Hi both,

Just tried out the most recent release and it makes a world of difference! Thanks for the update in such a timely manner! I think this might be a game changer...

If I may make another (hopefully small) request. Would it be possible to output the PC coefficients and explained % in a table format for easy cross comparison? At the moment, I'm pasting into Excel and having to delimit/copy-paste etc. Not a huge issue, but if you find some spare time, this might be helpful to others as well!

Cheers!

Jakub

Joe

JakubS

One way you can do this is by parsing the channel properties with python and copying that to the clipboard and pasting in Excel (or saving to Excel or csv). Here is my quick try at doing just that:

import re
import pandas as pd

pca_channels = [c for c in data.timeSeriesList(data.Output) if 'PCA' in c.name]
df = pd.DataFrame(index=[c.name for c in pca_channels])

for c in pca_channels:
	coeffs_str = [coeff.strip() for coeff in c.property('PCA Coeff.').split(',') if coeff.strip()] 
	coeffs = [re.match('([A-Z][a-z]?\d{1,3})\s\((-?\d+.\d+)\)', coeff).groups() for coeff in coeffs_str]

	if not df.columns.any():
		df[[c[0] for c in coeffs]] = None

	df.loc[c.name] = [float(c[1]) for c in coeffs]

df['Explained Var.'] = [float(c.property('PCA Explained Var.')) for c in pca_channels]
df.to_clipboard()

Create your PCA channels then run the above script in the python workspace. That'll create a table in your clipboard that you can paste directly into Excel.

If you find running the script tedious, you could wrap the whole thing up as a UI plugin to create a menu item and/or keyboard shortcut for it.

All the best,

JakubS

Thanks for the quick response, Joe! I think this script will work just fine. I do run into a minor snag which I can't seem to understand:

Evaluated at 2023-01-24 15:54:04.533
Traceback (most recent call last):
File "<string>", line 9, in <module>
File "<string>", line 9, in <listcomp>
AttributeError: 'NoneType' object has no attribute 'groups'

Complaining about line 9 here...

Edit: I found the issue; the regex should read: '([A-Z][a-z]?\d{1,3}\w+)\s((-?\d+.\d+))' with \w+ inserted to match the "_ppm"

Thanks again for all your help!

Joe

JakubS

Yup, that should do it. I was only testing it with input channels as the source, so only had ElementMass rather than ElementMass_ppm.