[Gwyddion-users] Problems with +pygwy install on MacOS 10.12.3 with Macports and compiling from source

[Austin Fox]

Hi All,

I would really like to script my daily SPM analysis but I can't seem to get
Gwyddion with pygwy to install.

*First Macports:*
<Error: org.macports.destroot for port gwyddion returned: error renaming
"/opt/local/var/macports/build/_opt_local_var_macports_sourc
es_rsync.macports.org_release_tarballs_ports_science_gwyddion/gwyddion/work/
destroot/opt/local/lib/python2.7/site-packages/gwy.so": no such file or
directory>

navigating to find files: bla/destroot/opt/local/lib/python2.7 does not
exist at all
ran: sudo port clean gwyddion so as to hopefully not effect compiling

*Compiling from source:*
No matter what options and or flags are set (I tried a bunch of stuff but
probably missed something key), output of ./configure gives:
< Python interface (pygwy): no (needs libpython2.7)>

libpython2.7.dylib does exist in /opt/local/lib
Attached is full output

*Other Notes/Thoughts:*
All dependancies are installed via Macports
Are these errors related? Is the issue stemming from not finding the
libpython2.7?
Am I just missing something entirely straight forward?

I have attempted to follow and try all that was in the the discussion in
the 2014 on pygwy via macports
<https://sourceforge.net/p/gwyddion/mailman/gwyddion-users/thread/20140427111002.GA10822%40physics.muni.cz/#msg32273589>
thread
but no luck.

Any help is appreciated.

Thank you,
Austin

Austin Fox
*Graduate Assistant*
*Multi-functional Thin Film Materials Group, Materials Science*
*School of Mechanical, Industrial, & Manufacturing Engineering*
*Oregon State University*
204 Rogers Hall | Corvallis, OR 97331 | Tel: *(717)-371-3077*
*www.openafox.com <http://www.openafox.com/>*

[David Nečas (Yeti)]

Post by Austin Fox
*Compiling from source:*
No matter what options and or flags are set (I tried a bunch of stuff but
< Python interface (pygwy): no (needs libpython2.7)>
...
Attached is full output

checking for python version... 2.7
checking for python platform... darwin
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages
checking size of pid_t... 4
checking for python build option BASECFLAGS... -fno-strict-aliasing -fno-common -dynamic
checking for python build option LDFLAGS... -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48
checking for python build option CCSHARED...
checking for python build option LINKFORSHARED... -u _PyMac_Error Python.framework/Versions/2.7/Python
checking for python build option LIBDIR... /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib

So, it finds Python, it finds out how it should compile and link with
libpython (it look reasonable to me, but I cannot tell if it is correct
– someone with working pygwy could maybe compare?)

checking for PyRun_String in -lpython2.7... no

but when it actually tries that by compiling a small test program it
fails. The detailed reason is in config.log around the line containing

checking for PyRun_String

You can look there, if the error message will make sense to you, or just
post the part of the file.

Post by Austin Fox
Are these errors related?

Probably.

Post by Austin Fox
Is the issue stemming from not finding the libpython2.7?

Not so much finding as using it.

Regards,

Yeti

[Austin Fox]

Hi Yeti,

Thanks for the help.

Here are the relevant lines:

configure:18027: checking for python build option LINKFORSHARED
configure:18048: result: -u _PyMac_Error Python.framework/Versions/2.7/
Python
configure:18065: checking for python build option LIBDIR
configure:18086: result: /opt/local/Library/Frameworks/
Python.framework/Versions/2.7/lib
configure:18117: checking for PyRun_String in -lpython2.7
configure:18142: gcc -o conftest -g -O2 conftest.c -lpython2.7
-L/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib
-L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 -u
_PyMac_Error Python.framework/Versions/2.7/Python >&5
clang: error: no such file or directory: 'Python.framework/Versions/2.
7/Python'
configure:18142: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "Gwyddion"
yatta yatta


All I can find is this similar problem:
http://stackoverflow.com/questions/6490513/vim-failing-
to-compile-with-python-on-os-x
but altering my pythons MakeFile seems to have no effect at all.

Thanks,
Austin

Post by David Nečas (Yeti)

Post by Austin Fox
*Compiling from source:*
No matter what options and or flags are set (I tried a bunch of stuff but
< Python interface (pygwy): no (needs libpython2.7)>
...
Attached is full output

checking for python version... 2.7
checking for python platform... darwin
checking for python script directory... ${prefix}/lib/python2.7/site-
packages
checking for python extension module directory...
${exec_prefix}/lib/python2.7/site-packages
checking size of pid_t... 4
checking for python build option BASECFLAGS... -fno-strict-aliasing -fno-common -dynamic
checking for python build option LDFLAGS... -L/opt/local/lib
-Wl,-headerpad_max_install_names -L/opt/local/lib/db48
checking for python build option CCSHARED...
checking for python build option LINKFORSHARED... -u _PyMac_Error
Python.framework/Versions/2.7/Python
checking for python build option LIBDIR... /opt/local/Library/Frameworks/
Python.framework/Versions/2.7/lib
So, it finds Python, it finds out how it should compile and link with
libpython (it look reasonable to me, but I cannot tell if it is correct
– someone with working pygwy could maybe compare?)
checking for PyRun_String in -lpython2.7... no
but when it actually tries that by compiling a small test program it
fails. The detailed reason is in config.log around the line containing
checking for PyRun_String
You can look there, if the error message will make sense to you, or just
post the part of the file.

Post by Austin Fox
Are these errors related?

Probably.

Post by Austin Fox
Is the issue stemming from not finding the libpython2.7?

Not so much finding as using it.
Regards,
Yeti
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by Austin Fox
configure:18027: checking for python build option LINKFORSHARED
configure:18048: result: -u _PyMac_Error Python.framework/Versions/2.7/Python
...
clang: error: no such file or directory: 'Python.framework/Versions/2.7/Python'

So this is what breaks it. After reading more on how to link with
libpython, we are probably not doing it right (along with lots of other
people). I will try to correct it.

But then I really really need people to test the change on OS X, i.e.
try to compile future development snapshots.

As an immediate help, override the LINKFORSHARED variable by running
configure as

./configure ... PYTHON_SYSCFG_LINKFORSHARED='-u _PyMac_Error /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python'

where ... stands for the usual arguments. If it does not fix it, at
least you should progress further and get a different error if there are
more problems.

Regards,

Yeti

[David Nečas (Yeti)]

Could you please run

python2.7-config --includes

and

python2.7-config --ldflags

and post the output to give me a better idea which OS X-specific
settings and workarounds may be necessary?

Thanks,

Yeti

[Austin Fox]

Post by David Nečas (Yeti)
Could you please run
python2.7-config --includes

-I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-I/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/include/python2.7and

Post by David Nečas (Yeti)
python2.7-config --ldflags

-L/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/lib/python2.7/config -lpython2.7 -ldl -framework CoreFoundation


Also I tried the work around and it works. It did not want to find pygtk
even though it is installed and importable in python. But after:
export
PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/pkgconfig/
as in the 2014 on pygwy via macports
<https://sourceforge.net/p/gwyddion/mailman/gwyddion-users/thread/20140427111002.GA10822%40physics.muni.cz/#msg32273589>
thread
it finds it.

Now when I make I get an error:

gwyshapefitpreset.c:1897:5: error: implicit declaration of function
'sincos' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
HANDLE_PHI_CACHE(phi);
^
gwyshapefitpreset.c:1098:13: note: expanded from macro 'HANDLE_PHI_CACHE'
sincos(phi, &sphi, &cphi); \
^
gwyshapefitpreset.c:1897:5: note: did you mean '__sincos'?
gwyshapefitpreset.c:1098:13: note: expanded from macro 'HANDLE_PHI_CACHE'
sincos(phi, &sphi, &cphi); \
^
/usr/include/math.h:666:29: note: '__sincos' declared here
__header_always_inline void __sincos(double __x, double *__sinp, double
*__cosp) {
^
1 error generated.
make[3]: *** [gwyshapefitpreset.lo] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Austin

[David Nečas (Yeti)]

Post by Austin Fox
-I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-I/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/include/python2.7and
-L/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/lib/python2.7/config -lpython2.7 -ldl -framework CoreFoundation

OK, this means I will try to just go with whatever python-config
provides and see if it works on OS X without adjustments. Tomorrow's
development snapshot (2.47.20170403) will already contain the rewritten
Python configuration.

Post by Austin Fox
gwyshapefitpreset.c:1897:5: error: implicit declaration of function
'sincos' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
HANDLE_PHI_CACHE(phi);

Now this is odd. Does configue say

checking for sincos... yes

? (You can also look for it in config.log). Is there line

#define HAVE_SINCOS 1

in config.h?

I do not understand it in either case but for start I'd like to know if
we found sincos and then mysteriously lost it, or we did not find sincos
and are trying to use it anyway (this would be a Gwyddion bug, but I
checked gwyshapefitpreset.c and it seems correct).

Or maybe first please try the compilation from a freshly unpacked source
code. I suspect stale files or something similar.

Regards,

Yeti

[Austin Fox]

Sorry I should have included that.

checking for sincos... no

in log:
configure:22205: checking for sincos
configure:22205: gcc -o conftest -g -O2 -I/usr/X11/include -L/usr/X11/lib
conftest.c >&5
Undefined symbols for architecture x86_64:
"_sincos", referenced from:
_main in conftest-694f52.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
configure:22205: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "Gwyddion"
...
| /* end confdefs.h. */
| /* Define sincos to an innocuous variant, in case <limits.h> declares
sincos.
| For example, HP-UX 11i <limits.h> declares gettimeofday. */
| #define sincos innocuous_sincos
|
| /* System header to define __stub macros and hopefully few prototypes,
| which can conflict with char sincos (); below.
| Prefer <limits.h> to <assert.h> if __STDC__ is defined, since
| <limits.h> exists even on freestanding compilers. */
|
| #ifdef __STDC__
| # include <limits.h>
| #else
| # include <assert.h>
| #endif
|
| #undef sincos
|
| /* Override any GCC internal prototype to avoid an error.
| Use char because int might match the return type of a GCC
| builtin and then its argument prototype would still apply. */
| #ifdef __cplusplus
| extern "C"
| #endif
| char sincos ();
| /* The GNU C library defines this for functions which it implements
| to always fail with ENOSYS. Some functions are actually named
| something starting with __ and the normal name is an alias. */
| #if defined __stub_sincos || defined __stub___sincos
| choke me
| #endif
|
| int
| main ()
| {
| return sincos ();
| ;
| return 0;
| }
configure:22205: result: no

In config.h:
/* Define to 1 if you have the `sincos' function. */
/* #undef HAVE_SINCOS */


So still trying to use sincos..??

****
Now if I use the dev snapshot instead of the stable version I no longer
have the sincos error but the same issue as the 2014 on pygwy via macports
<https://sourceforge.net/p/gwyddion/mailman/gwyddion-users/thread/20140427111002.GA10822%40physics.muni.cz/#msg32273589>
thread.
With , on make:
pygwy.c:31:10: fatal error: 'pygtk-2.0/pygobject.h' file not found

That is with conf:
./configure PYTHON_SYSCFG_LINKFORSHARED='-u _PyMac_Error
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python'
PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/pkgconfig/
DYLD_FALLBACK_LIBRARY_PATH="/opt/local/lib"
CPPFLAGS="-I/opt/local/include-I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/"
LDFLAGS="-L/opt/local/lib"

Austin

Post by Austin Fox

Post by Austin Fox
-I/opt/local/Library/Frameworks/Python.framework/

Versions/2.7/include/python2.7

Post by Austin Fox
-I/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/include/python2.7and
-L/opt/local/Library/Frameworks/Python.framework/
Versions/2.7/lib/python2.7/config -lpython2.7 -ldl -framework

CoreFoundation
OK, this means I will try to just go with whatever python-config
provides and see if it works on OS X without adjustments. Tomorrow's
development snapshot (2.47.20170403) will already contain the rewritten
Python configuration.

Post by Austin Fox
gwyshapefitpreset.c:1897:5: error: implicit declaration of function
'sincos' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
HANDLE_PHI_CACHE(phi);

Now this is odd. Does configue say
checking for sincos... yes
? (You can also look for it in config.log). Is there line
#define HAVE_SINCOS 1
in config.h?
I do not understand it in either case but for start I'd like to know if
we found sincos and then mysteriously lost it, or we did not find sincos
and are trying to use it anyway (this would be a Gwyddion bug, but I
checked gwyshapefitpreset.c and it seems correct).
Or maybe first please try the compilation from a freshly unpacked source
code. I suspect stale files or something similar.
Regards,
Yeti
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by Austin Fox
Now if I use the dev snapshot instead of the stable version I no longer
have the sincos error

Ah, sorry. There was a indeed bug in 2.47 (unguarded sincos()) – I was
checking the current source code where it is already corrected.

Post by Austin Fox
but the same issue as the 2014 on pygwy via macports
<https://sourceforge.net/p/gwyddion/mailman/gwyddion-users/thread/20140427111002.GA10822%40physics.muni.cz/#msg32273589>
thread.
pygwy.c:31:10: fatal error: 'pygtk-2.0/pygobject.h' file not found
./configure PYTHON_SYSCFG_LINKFORSHARED='-u _PyMac_Error
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python'
PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/pkgconfig/
DYLD_FALLBACK_LIBRARY_PATH="/opt/local/lib"
CPPFLAGS="-I/opt/local/include-I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/"
LDFLAGS="-L/opt/local/lib"

DYLD_FALLBACK_LIBRARY_PATH – I still do not know what to do about it.

PYTHON_SYSCFG_LINKFORSHARED – has no longer any effect; you can drop it.

CPPFLAGS – a space is missing before the second -I, at least that is how
it is formatted in the e-mail. You should certainly not need the second
part; we should be getting the Python include path from ‘python-config
--includes’. Although the path is different: you do not have the
‘python2.7’ at the end there.

Regarding the pygobject error, there is indeed a bug (and has been there
forever!): the ‘pygtk-2.0/’ does not belong there. Please see the
attached patch.

I am glad someone started to complain – otherwise the problems would
never be fixed.

Regards,

Yeti

[Austin Fox]

Thank you Yeti!

I found/had one other issue in gwy.c
line 89
gchar *filename = g_strconcat(gwyddion_libs[i],
".", GWY_SHARED_LIBRARY_EXTENSION,
".0",
NULL);
the ".0" either should not be there or should be before the GWY_SHA... as
it causes the libgwy not to load b/c of wrong file name.

delete the ,".0" and gwy now imports!


Here is the install process for anyone who needs it:

1. Download <http://gwyddion.net/download.php#head> and extract
<http://gwyddion.net/documentation/user-guide-en/installation-unix-source.html>
latest dev snapshot
2. Apply patches:
a. gwyddion-2.47-pygobject-include.patch Attached - copy into extracted
gwyddion folder and run:
patch -p0 < gwyddion-2.47-pygobject-include.patch
b. Open gwy.c and delete , ".0" in line 90
3. run config
./configure PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.
framework/Versions/2.7/lib/pkgconfig/ DYLD_FALLBACK_LIBRARY_PATH="/
opt/local/lib" CPPFLAGS="-I/opt/local/include -I/opt/local/Library/
Frameworks/Python.framework/Versions/2.7/include/"
LDFLAGS="-L/opt/local/lib"
4. make install (I needed sudo)
5. export PYTHONPATH=/usr/local/lib/python2.7/site-packages (I added this
to my .bash_profile)
6. Either invoke gwyddion or import gwy in python


Also - I am assuming this is normal but just incase it is not I get 3
duplicate warnings on import:
** (<unknown>:25137): WARNING **: Trying to register gtype
'GMountMountFlags' as enum when in fact it is of type 'GFlags'

Cheers,
Austin

Post by David Nečas (Yeti)

Post by Austin Fox
Now if I use the dev snapshot instead of the stable version I no longer
have the sincos error

Ah, sorry. There was a indeed bug in 2.47 (unguarded sincos()) – I was
checking the current source code where it is already corrected.

Post by Austin Fox
but the same issue as the 2014 on pygwy via macports
<https://sourceforge.net/p/gwyddion/mailman/gwyddion-

users/thread/20140427111002.GA10822%40physics.muni.cz/#msg32273589>

Post by Austin Fox
thread.
pygwy.c:31:10: fatal error: 'pygtk-2.0/pygobject.h' file not found
./configure PYTHON_SYSCFG_LINKFORSHARED='-u _PyMac_Error
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python'
PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.

framework/Versions/2.7/lib/pkgconfig/

Post by Austin Fox
DYLD_FALLBACK_LIBRARY_PATH="/opt/local/lib"
CPPFLAGS="-I/opt/local/include-I/opt/local/Library/

Frameworks/Python.framework/Versions/2.7/include/"

Post by Austin Fox
LDFLAGS="-L/opt/local/lib"

DYLD_FALLBACK_LIBRARY_PATH – I still do not know what to do about it.
PYTHON_SYSCFG_LINKFORSHARED – has no longer any effect; you can drop it.
CPPFLAGS – a space is missing before the second -I, at least that is how
it is formatted in the e-mail. You should certainly not need the second
part; we should be getting the Python include path from ‘python-config
--includes’. Although the path is different: you do not have the
‘python2.7’ at the end there.
Regarding the pygobject error, there is indeed a bug (and has been there
forever!): the ‘pygtk-2.0/’ does not belong there. Please see the
attached patch.
I am glad someone started to complain – otherwise the problems would
never be fixed.
Regards,
Yeti
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gwyddion-users mailing list
https://lists.sourceforge.net/lists/listinfo/gwyddion-users

[David Nečas (Yeti)]

Post by Austin Fox
I found/had one other issue in gwy.c
line 89
gchar *filename = g_strconcat(gwyddion_libs[i],
".", GWY_SHARED_LIBRARY_EXTENSION,
".0",
NULL);
the ".0" either should not be there or should be before the GWY_SHA... as
it causes the libgwy not to load b/c of wrong file name.
delete the ,".0" and gwy now imports!

Hm. So the libraries are called like libgwyddion2.0.dylib? This
differs from Linux and BSD (possibly other ELF systems, not sure here)
where the version appears after the extension: libgwyddion2.so.0.

The unnumbered library (which may be just a link) belongs to the
development package on systems where packages are split into user and
development parts (e.g. most Linux distros). Removing ".0" would make
the Python extension module require the development package.

I can add an exception for Dawrin as a quick fix but a better solution
is to get the library names from the configuration. Could you please
send me your libtool (should be created in top-level directory by
configure).

Post by Austin Fox
a. gwyddion-2.47-pygobject-include.patch

Should be no longer necessary.

Post by Austin Fox
3. run config
./configure PKG_CONFIG_PATH=/opt/local/Library/Frameworks/Python.
framework/Versions/2.7/lib/pkgconfig/ DYLD_FALLBACK_LIBRARY_PATH="/
opt/local/lib" CPPFLAGS="-I/opt/local/include -I/opt/local/Library/
Frameworks/Python.framework/Versions/2.7/include/"
LDFLAGS="-L/opt/local/lib"

Did you try without

-I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/

in CPPFLAGS?

Post by Austin Fox
Also - I am assuming this is normal but just incase it is not I get 3
** (<unknown>:25137): WARNING **: Trying to register gtype
'GMountMountFlags' as enum when in fact it is of type 'GFlags'

Yes, this is ‘normal’ and we cannot do much about it because it occurs
in GLib.

Regards,

Yeti

[Robb Bean]

Hi Yeti,

Post by David Nečas (Yeti)

Post by Austin Fox
I found/had one other issue in gwy.c
line 89
gchar *filename = g_strconcat(gwyddion_libs[i],
".", GWY_SHARED_LIBRARY_EXTENSION,
".0",
NULL);
the ".0" either should not be there or should be before the GWY_SHA... as
it causes the libgwy not to load b/c of wrong file name.
delete the ,".0" and gwy now imports!

Hm. So the libraries are called like libgwyddion2.0.dylib?

It's quite some time I worked on a Mac but as far as I remember this is
the case – the version numbers are in front of the file name extension.

Best regards,
Robb

[David Nečas (Yeti)]

Post by Robb Bean
It's quite some time I worked on a Mac but as far as I remember this is
the case – the version numbers are in front of the file name extension.

Thanks for the confirmation.

In today's development snapshot the workarounds and variables you need
to set manually have been hopefully reduced to the usual ones (i.e. may
need to be set in general in manual installation, not just on OS X).

Regards,

Yeti