[Gwyddion-users] cannot load/save file with pygwy in python console outside gwyddion

[YTC]

I wrote some scripts to batch process files in a python console outside
gwyddion on OSX. But it failed to load files.

c=gwy.gwy_file_load(fp, gwy.RUN_NONINTERACTIVE)
Error Output:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/wx-3.0-osx_cocoa/wx/py/PyCrust.py:1:
Warning: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
#!/usr/bin/env python

And it cannot save as well. I have a separate module to load a custom data
file.

c=file_load() # returns a gwy container
......
data processing
......
gwy.gwy_file_save(c, 'test.gwy', gwy.RUN_NONINTERACTIVE)

Traceback (most recent call last):
File "<input>", line 1, in <module>
GError: No module can save to this file type.
executing the last line will crash the python console.

Both scripts work fine in windows, and in the built-in pygwy console on
Mac. I suspect it might be problems with loading the gwyddion libraries on
OSX, but have no ideas on how to track down the problem. Any help is much
appreciated.

[David Nečas (Yeti)]

Post by YTC
I wrote some scripts to batch process files in a python console outside
gwyddion on OSX. But it failed to load files.

Please test this with a development snapshot of Gwyddion. There were
some changes in the standalone gwy Python module since 2.36. They
should influence dlopening of libraries, not modules, but have to make
sure things work in what will become 2.37.

Post by YTC

import gwy
gwy.gwy_module_browser()

This is what I suggest to do if you suspect modules not being loaded.

Regards,

Yeti

[YTC]

Hi Yeti,

I tried to install the development snapshot using Nicola Ferralis's script.
But it failed at the end. Here is the last few lines about the error. The
mentioned log file does not exist. My current version was installed using
Macports with "+pygwy" option. Not sure if this extra option is causing the
problem.


---> Configuring gwyddion

---> Building gwyddion

Error: org.macports.build for port gwyddion returned: command execution
failed

Please see the log file for port gwyddion for details:


/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_gwyddion/gwyddion/main.log

To report a bug, follow the instructions in the guide:

http://guide.macports.org/#project.tickets

Error: Processing of port gwyddion failed

---> Cleaning gwyddion


Done!


I also tried to run the module browser from python shell. It opens up a
blank window. This happens in both OSX and Windows 8. (Even though the
issue I had in my original post is for OSX only.)

Any suggestions?

-YTC

Post by David Nečas (Yeti)

Post by YTC
I wrote some scripts to batch process files in a python console outside
gwyddion on OSX. But it failed to load files.

Please test this with a development snapshot of Gwyddion. There were
some changes in the standalone gwy Python module since 2.36. They
should influence dlopening of libraries, not modules, but have to make
sure things work in what will become 2.37.

Post by YTC

import gwy
gwy.gwy_module_browser()

This is what I suggest to do if you suspect modules not being loaded.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
I tried to install the development snapshot using Nicola Ferralis's script.
But it failed at the end. Here is the last few lines about the error. The
mentioned log file does not exist.

Unfortunately, the messages it printed are not partricularly informative
so I cannot tell what happened.

Post by YTC
I also tried to run the module browser from python shell. It opens up a
blank window. This happens in both OSX and Windows 8. (Even though the
issue I had in my original post is for OSX only.)
Any suggestions?

Ah, I was doing it from the command line. Apparently if you do that
from IDLE or something similar (an IDE/graphical shell) there can be
some bad interactions between the Gtk+ and native GUI event loops
running in the same program. In WinXP the module browser window was
non-responsive when I tried it in IDLE. But using the plain command
line python shell and running

import os
os.chdir(r'c:\program files\gwyddion\bin')
import gwy
gwy.gwy_module_browser()

worked fine for me. Note the chdir is a dirty shortcut for setting up
paths correctly.

In OS X it may be a similar problem. Is the module browser completely
blank in OS X or you get an empty list?

Regards,

Yeti

[David Nečas (Yeti)]

Post by YTC
Any suggestions?

Alternatively, do not try to make the full browser work, just run

print gwy.gwy_module_get_filename('gwyfile')

It should either print the on-disk file from which the ‘gwyfile’
Gwyddion module was loaded or produce a warning to the shell and print
‘None’.

Regards,

Yeti

[YTC]

print gwy.gwy_module_get_filename('gwyfile') retuns none. Cross checked on
windows and it returns the correct path to gwyddion/lib/modules.

I tried to add /opt/local/lib/gwyddion to LD_LIBRARY_PATH on OSX, which
didn't solve the problem. So probably not the correct path and/or variable
to set?

Post by David Nečas (Yeti)

Post by YTC
Any suggestions?

Alternatively, do not try to make the full browser work, just run
print gwy.gwy_module_get_filename('gwyfile')
It should either print the on-disk file from which the ‘gwyfile’
Gwyddion module was loaded or produce a warning to the shell and print
‘None’.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
print gwy.gwy_module_get_filename('gwyfile') retuns none. Cross checked on
windows and it returns the correct path to gwyddion/lib/modules.
I tried to add /opt/local/lib/gwyddion to LD_LIBRARY_PATH on OSX, which
didn't solve the problem. So probably not the correct path and/or variable
to set?

The libraries are definitely found if you can do ‘import gwy’, only
modules are the problem. Try

print gwy.gwy_find_self_dir('modules')

Is it what you expect? If not, it may help setting GWYDDION_LIBDIR, see

http://gwyddion.net/documentation/user-guide-en/gwyddion.html#gwyddion-environment

IIRC OS X packages may need to fiddle with these too.

Regards,

Yeti

[YTC]

print gwy.gwy_find_self_dir('modules') returns

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/modules

which is expected to be /opt/local/lib. I have no idea how that path comes
about.

I found that link about GWYDDION_LIBDIR earlier too. But setting it to
/opt/local/lib doesn't work.

Post by YTC

Post by YTC
print gwy.gwy_module_get_filename('gwyfile') retuns none. Cross checked

on

Post by YTC
windows and it returns the correct path to gwyddion/lib/modules.
I tried to add /opt/local/lib/gwyddion to LD_LIBRARY_PATH on OSX, which
didn't solve the problem. So probably not the correct path and/or

variable

Post by YTC
to set?

The libraries are definitely found if you can do ‘import gwy’, only
modules are the problem. Try
print gwy.gwy_find_self_dir('modules')
Is it what you expect? If not, it may help setting GWYDDION_LIBDIR, see
http://gwyddion.net/documentation/user-guide-en/gwyddion.html#gwyddion-environment
IIRC OS X packages may need to fiddle with these too.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
print gwy.gwy_find_self_dir('modules') returns
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/modules

This is a very very strange path. It would mean Gwyddion modules are
installed to some Python module directory. What do you get when you try
the same in pygwy console inside Gwyddion?

Post by YTC
I found that link about GWYDDION_LIBDIR earlier too. But setting it to
/opt/local/lib doesn't work.

Did you export it? I.e. does gwy.gwy_find_self_dir('modules') print an
actual Gwyddion module directory after ‘import gwy’, but still no
modules are loaded?

Regards,

Yeti

[YTC]

it works fine in the pygwy console (returns /opt/local/lib/gwyddion/modules)

I did export it. This is confirmed with os.environ['GWYDDION_LIBDIR'] in
python shell.
With this, gwy.gwy_find_self_dir('modules') still returns that previous path

Post by YTC

Post by YTC
print gwy.gwy_find_self_dir('modules') returns

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/modules
This is a very very strange path. It would mean Gwyddion modules are
installed to some Python module directory. What do you get when you try
the same in pygwy console inside Gwyddion?

Post by YTC
I found that link about GWYDDION_LIBDIR earlier too. But setting it to
/opt/local/lib doesn't work.

Did you export it? I.e. does gwy.gwy_find_self_dir('modules') print an
actual Gwyddion module directory after ‘import gwy’, but still no
modules are loaded?
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[YTC]

btw, when I import gwy in python shell
it returns a warning

Gwydgets-WARNING **: Cannot open directory
`/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/pixmaps':
Error opening directory
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/pixmaps':
No such file or directory

The pixmaps is expected to be in /opt/local/share/gwddion, but somehow it
also points to that strange Python.app path.

Post by YTC
it works fine in the pygwy console (returns
/opt/local/lib/gwyddion/modules)
I did export it. This is confirmed with os.environ['GWYDDION_LIBDIR'] in
python shell.
With this, gwy.gwy_find_self_dir('modules') still returns that previous path

Post by YTC

Post by YTC
print gwy.gwy_find_self_dir('modules') returns

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Resources/modules
This is a very very strange path. It would mean Gwyddion modules are
installed to some Python module directory. What do you get when you try
the same in pygwy console inside Gwyddion?

Post by YTC
I found that link about GWYDDION_LIBDIR earlier too. But setting it to
/opt/local/lib doesn't work.

Did you export it? I.e. does gwy.gwy_find_self_dir('modules') print an
actual Gwyddion module directory after ‘import gwy’, but still no
modules are loaded?
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
I did export it. This is confirmed with os.environ['GWYDDION_LIBDIR'] in
python shell.
With this, gwy.gwy_find_self_dir('modules') still returns that previous path

Ah, I see now. There is a special self-finding code for Apple systems
that locates the library directory in the main bundle

https://sourceforge.net/p/gwyddion/code/HEAD/tree/trunk/gwyddion/libgwyddion/gwyutils.c#l559

Which is the core of the problem. When you run Gwyddion the main bundle
is Gwyddion and you get the Gwyddion directory. But when you run Python
the main bundle is Python and you get the Python directory instead. I
will give the environment variables the highest priority on all systems;
this is how they should work anyway.

It should be also possible to fix this by passing the Gwyddion bundle to
CFBundleCopyResourcesDirectoryURL() and CFBundleCopyBundleURL() instead
of whatever we get from CFBundleGetMainBundle(). However, I do not know
how to get ‘the Gwyddion bundle’. I suppose using
CFBundleGetBundleWithIdentifier() with the correct bundle ID. What is
the correct bundle ID for Gwyddion? I have no idea.

Regards,

Yeti

[YTC]

in gwyutils.c there is the following code

#ifdef __APPLE__
char *ret = gwy_osx_find_dir_in_bundle(dirname);
if (ret)
return ret;
#endif /* __APPLE__ */

It seems gwy_find_self_dir will return here instead of the values defined
by the environmental variables.

Post by David Nečas (Yeti)

Post by YTC
I did export it. This is confirmed with os.environ['GWYDDION_LIBDIR'] in
python shell.
With this, gwy.gwy_find_self_dir('modules') still returns that previous

path
Ah, I see now. There is a special self-finding code for Apple systems
that locates the library directory in the main bundle
https://sourceforge.net/p/gwyddion/code/HEAD/tree/trunk/gwyddion/libgwyddion/gwyutils.c#l559
Which is the core of the problem. When you run Gwyddion the main bundle
is Gwyddion and you get the Gwyddion directory. But when you run Python
the main bundle is Python and you get the Python directory instead. I
will give the environment variables the highest priority on all systems;
this is how they should work anyway.
It should be also possible to fix this by passing the Gwyddion bundle to
CFBundleCopyResourcesDirectoryURL() and CFBundleCopyBundleURL() instead
of whatever we get from CFBundleGetMainBundle(). However, I do not know
how to get ‘the Gwyddion bundle’. I suppose using
CFBundleGetBundleWithIdentifier() with the correct bundle ID. What is
the correct bundle ID for Gwyddion? I have no idea.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
in gwyutils.c there is the following code
#ifdef __APPLE__
char *ret = gwy_osx_find_dir_in_bundle(dirname);
if (ret)
return ret;
#endif /* __APPLE__ */
It seems gwy_find_self_dir will return here instead of the values defined
by the environmental variables.

Yes, I have already fixed this, see

https://sourceforge.net/p/gwyddion/code/16309/

According to the Apple documentation and

https://trac.macports.org/browser/trunk/dports/science/gwyddion/files/Info.plist

the correct bundle indentifier for Gwyddion seems to be "net.gwyddion".
However, I will only try to change the Apple-specific self-finding code
if someone is able to actually compile and test the change since I
cannot test anything on OS X.

Regards,

Yeti

[YTC]

If you can send me gwyutils.c with the corrected bundle identifier, I will
be happy to test it.

Post by David Nečas (Yeti)

Post by YTC
in gwyutils.c there is the following code
#ifdef __APPLE__
char *ret = gwy_osx_find_dir_in_bundle(dirname);
if (ret)
return ret;
#endif /* __APPLE__ */
It seems gwy_find_self_dir will return here instead of the values defined
by the environmental variables.

Yes, I have already fixed this, see
https://sourceforge.net/p/gwyddion/code/16309/
According to the Apple documentation and
https://trac.macports.org/browser/trunk/dports/science/gwyddion/files/Info.plist
the correct bundle indentifier for Gwyddion seems to be "net.gwyddion".
However, I will only try to change the Apple-specific self-finding code
if someone is able to actually compile and test the change since I
cannot test anything on OS X.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
If you can send me gwyutils.c with the corrected bundle identifier, I will
be happy to test it.

I would try this, I cannot even promise it will compile though...

Regards,

Yeti

[YTC]

I have tested this patch with 20140619 snapshot. It compiled successfully.
I then removed all the environment variables. Everything still works
nicely. Haven't had any problems so far.

Thanks a lot, Yeti! This really makes my life much easier!

Post by David Nečas (Yeti)

Post by YTC
If you can send me gwyutils.c with the corrected bundle identifier, I

will

Post by YTC
be happy to test it.

I would try this, I cannot even promise it will compile though...
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by YTC
I have tested this patch with 20140619 snapshot. It compiled successfully.
I then removed all the environment variables. Everything still works
nicely. Haven't had any problems so far.

Good, I committed the change (16312) so it will be in 2.37.

Regards,

Yeti

[YTC]

I compiled with the gwyutils.c file from 16309, and now everything works
great. Thanks so much!

Post by David Nečas (Yeti)

Post by YTC
in gwyutils.c there is the following code
#ifdef __APPLE__
char *ret = gwy_osx_find_dir_in_bundle(dirname);
if (ret)
return ret;
#endif /* __APPLE__ */
It seems gwy_find_self_dir will return here instead of the values defined
by the environmental variables.

Yes, I have already fixed this, see
https://sourceforge.net/p/gwyddion/code/16309/
According to the Apple documentation and
https://trac.macports.org/browser/trunk/dports/science/gwyddion/files/Info.plist
the correct bundle indentifier for Gwyddion seems to be "net.gwyddion".
However, I will only try to change the Apple-specific self-finding code
if someone is able to actually compile and test the change since I
cannot test anything on OS X.
Regards,
Yeti
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users