Action Item #457
Create backend libraries
| Status: | Closed | Start date: | 12 Jul 2015 | |
|---|---|---|---|---|
| Priority: | Immediate | Due date: | ||
| Assignee: |  Hammel | % Done: | 100% | |
| Category: | - | |||
| Target version: | PiBox - 0.11.0 | |||
| Severity: | 01 - Critical |
Description
This is the start of the API layer in PiBox.
Libraries to create:- Network (from piboxd, bui-network-config)
- Include smb support
- Add bluetooth config
- config.txt handler - changing RPi configuration such as audio output and hdmi setup
- App messaging (see RM #433)
More to come.
See- examples of how to create libraries within programs (like with bui-network-config).
- Whose afraid of autotools (better example)
- How to write shared libraries
- Good practice in library design
- Program library howto - shared libraries
Associated revisions
RM #457: First step in creating a library for bui-network-config: clean up the build so it's generated the same as
all other PiBox packages and create an opkg for it (so it's easier to update).
RM #457: First step in creating a library for bui-network-config: clean up the build so it's generated the same as
all other PiBox packages and create an opkg for it (so it's easier to update).
RM #457: Quiet complaints from autoconf on build. Cleanup generated files on distclean.
RM #457: Quiet complaints from autoconf on build. Cleanup generated files on distclean.
RM #457: Integrate libtool use.
RM #457: Integrate libtool use.
RM #457: Integrate libtool use to generate shared library for load/save functions.
RM #457: Integrate libtool use to generate shared library for load/save functions.
RM #457: Make sure all installed files are properly owned.
RM #457: Make sure all installed files are properly owned.
RM #457: Set include install directory to buinet. Make sure cross tools are available to install processing.
RM #457: Set include install directory to buinet. Make sure cross tools are available to install processing.
RM #457: Change package name to buinet so headers get put into a decent directory.
RM #457: Change package name to buinet so headers get put into a decent directory.
RM #457: Converted from bui- to pibox- and changed the library prefix to pnc.
RM #457: Converted from bui- to pibox- and changed the library prefix to pnc.
RM #457: Add --version-info to library naming.
RM #457: Add --version-info to library naming.
RM #457: Change all references to bui-network-config to pibox-network-config.
RM #457: Change all references to bui-network-config to pibox-network-config.
RM #457: Fix header directory name for library in packaging.
RM #457: Fix header directory name for library in packaging.
RM #457: More migration to a proper library API: merged save.h and load.h to pnc.h (public API) and pncPrivate.h (internal API) and migrated all data structure access in those modules to get/set functions. Use libtools features to export only regex specified functions.
RM #457: More migration to a proper library API: merged save.h and load.h to pnc.h (public API) and pncPrivate.h (internal API) and migrated all data structure access in those modules to get/set functions. Use libtools features to export only regex specified functions.
RM #457: Renamed upstream project to pibox/pibox-network-config and adjusted cdtools accordingly.
RM #457: Renamed upstream project to pibox/pibox-network-config and adjusted cdtools accordingly.
RM #457: Updated bui-network-config package to build with library from cleaned up upstream repo.
RM #457: Migrate to pibox-network-config from bui-network-config.
RM #457: Convert to a properly built library package, with proper library API structure. This is an initial migration from app-specific functions such as logging and utilities to a common library.
RM #457: Fix deprecated reference to loggerLocal to use new library naming prefix.
RM #457: Ported to use libpibox's logger service.
RM #457: Ported to use libpibox's logger service.
RM #457: Added libpibox to development platform build.
RM #457: Add setter/getter for log verbosity level.
RM #457: Corrected call to get verbosity API function in DBG macros.
RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.
RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.
RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.
RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.
RM #457: Switched to using g_strdup from strdup.
RM #457: Switched to using g_strdup from strdup.
RM #457: Mirgrate stripNewline() calls to use libpibox pncStripNewline().
RM #457: Mirgrate stripNewline() calls to use libpibox pncStripNewline().
RM #457: Remove all references to local utils/log modules in favor of libpibox versions.
Changed pncUpdateInterface() to take addr, netmask and gateway arguments instead of calling back to the UI to get them. Caller now makes that UI call before calling pncUpdateInterface.
Removed UI callbacks from network library. Callers now handle UI callbacks before calling library API functions.
RM #457: Remove all references to local utils/log modules in favor of libpibox versions.
Changed pncUpdateInterface() to take addr, netmask and gateway arguments instead of calling back to the UI to get them. Caller now makes that UI call before calling pncUpdateInterface.
Removed UI callbacks from network library. Callers now handle UI callbacks before calling library API functions.
RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.
RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.
RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.
RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.
RM #457: Remove gdk dependency from network library.
RM #457: Remove gdk dependency from network library.
RM #457: Initial porting to libpibox and libpiboxnet. Still much work to be done in the port to libpiboxnet.
RM #457: Added pncSetInterfaceField() to libpiboxnet API.
RM #457: Added pncSetInterfaceField() to libpiboxnet API.
RM #457: Add missing clear and remove functions to API needed by piboxd.
RM #457: Add missing clear and remove functions to API needed by piboxd.
RM #457: Add missing reference to glib.h in pnc.h so library users don't need to.
RM #457: Add missing reference to glib.h in pnc.h so library users don't need to.
RM #457: Remove return value from pncSetInterfaceField. Check for invalid idx in get/setHostAPField().
RM #457: Remove return value from pncSetInterfaceField. Check for invalid idx in get/setHostAPField().
RM #457: Fix prototype for pncLoggerInit() to include gchar pointer parameter.
RM #457: Pass log file name (if any) to pncLoggerInit().
RM #457: Pass log file name (if any) to pncLoggerInit().
RM #457: Change from gchar to char in prototype for pncLoggerInit to match function.
RM #457: Fixed leaks exposed by Valgrind. Fixed errors in startup not exposed before switch to library that have to do with using uninitialized variables in UI. Switched to use static library for pibox-network-config, possibly only for initial testing.
RM #457: Fixed leaks exposed by Valgrind. Fixed errors in startup not exposed before switch to library that have to do with using uninitialized variables in UI. Switched to use static library for pibox-network-config, possibly only for initial testing.
RM #457: There were many mistakes in the port to the pnc library. These updates fix all known errors and allow pibox-network-config to properly generate data files under test mode. Removed glade files - this is no longer managed by glade. If you want UI updates, you gotta do it by hand now.
RM #457: There were many mistakes in the port to the pnc library. These updates fix all known errors and allow pibox-network-config to properly generate data files under test mode. Removed glade files - this is no longer managed by glade. If you want UI updates, you gotta do it by hand now.
RM #457: Add debug symbols to library to make it easier to debug with gdb.
RM #457: Add pncToLower() function.
RM #457: Revert to shared library use for pibox-network-config from static library used in previous debugging.
RM #457: Revert to shared library use for pibox-network-config from static library used in previous debugging.
RM #457: Migrate from archive download to git download for pibox-network-config repo.
RM #457: Cleanup compile time warnings and errors introduced when upgrading to gcc 5.1.
RM #457: Cleanup compile time warnings and errors introduced when upgrading to gcc 5.1.
RM #457: Add check for null argument to pncNetInterfaceValid().
RM #457: Add check for null argument to pncNetInterfaceValid().
RM #457: Add missing prototype for pncStripNewline to utils.h. Add missing header reference for ctype.h to utilc.c.
RM #457: Prevent segfaults if pncLogger is called with null format string.
RM #457: Fix potential segfaults by testing input arguments to callbacks for list handlers.
RM #457: Fix potential segfaults by testing input arguments to callbacks for list handlers.
RM #457: Remove extraneous file.
RM #457: Extend suppression file to ignore gobject, libpixman and a few others that are not part of the code under test.
RM #457: Extend test to support running with valgrind, split out SIPV4 subtests to be callable individually and report pass/fail, launch piboxd and kill it automatically when running the tests. All of this is part of the port of piboxd to libpiboxnet.
RM #457: Completed port for libpiboxnet by applying fixes (mostly in the new libraries: libpiboxnet and libpixbox) and removing old code that is now in the libraries. All functionality passes the server tests.
History
#1
Updated by Hammel about 2 years ago
- Project changed from PiBox to pibox-network-config
- Category deleted (
04 - Applications)
#2
Updated by Hammel about 2 years ago
- Subject changed from Move bui-network-config backend into a library to Create backend libraries
- Description updated (diff)
#3
Updated by Hammel about 2 years ago
- Description updated (diff)
#4
Updated by Hammel about 2 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
Restructured bui-network-config's build so it's just like all other PiBox builds (using autoreconf -i to gen the configure script). Also added opkg packaging to it. This will make it easier to test the migration to a library + frontend from the stand alone app.
Now I need to verify that the core builds this correctly.
#5
Updated by Hammel about 2 years ago
- % Done changed from 10 to 20
bui-network-config build verified working with existing structure of pibox dev platform build system. No changes required.
Now to break bui-network-config into a core and a shared library.
#6
Updated by Hammel about 2 years ago
- Description updated (diff)
#7
Updated by Hammel about 2 years ago
I've created a shared lib and the local package-build works fine. Everything gets installed where it should (/usr/include/buinet for headers, /usr/lib for libs). But when I run it from Buildroot it doesn't complete correctly. The libs and program get installed fine. The headers are missing. The log says they got installed, but they aren't there. No idea why yet.
#8
Updated by Hammel about 2 years ago
The command to install the headers works if I run it manually. So maybe there is something weird about rerunning the buildroot build? Maybe I need to rerun the whole thing. Yuck. I shouldn't have to do that.
I'll rerun the build tonight from scratch, just to see if that has any affect.
#9
Updated by Hammel about 2 years ago
- Description updated (diff)
#10
Updated by Hammel about 2 years ago
- Description updated (diff)
#11
Updated by Hammel about 2 years ago
I've changed the name of the program to pibox-network-config and the prefix to pnc for all library routines. This builds fine. I'll commit and push.
Then I need to create an init.c that will be part of the library. It's purpose is to abstract the filenames and provide get/set functions for them. I'll also need to make INTERFACE_T abstract so its members can only be retrieved via a get function (via GList's).
- Once the init module is written then I can implement CLI handling for specifying the filenames when testing.
- The bui-network-config package will need to change to pibox-network-config.
- The upstream repo should be changed from bui-network-config to pibox-network-config
utils.c and log.c need to be merged into libPibox. That means libPibox will need to be part of the development platform build.
#12
Updated by Hammel about 2 years ago
bui-network-config has now been changed to pibox-network-config in the source tree and in the upstream repo. The repo was also moved out from the xbui parent to the pibox parent. The source tree now builds the pibox-network-config binary along with a shared library and public API headers. I removed all direct references to private structures in the library from the application and switched to using only getter/setter methods. Everything is tested on the target and seems to be working correctly. I've even added opkg support to the build so future updates don't require rebuilding the rootfs.
So now I'm updating the core development platform to use the updated package and repo names. This will take a while because I had to wipe the existing rootfs build to properly verify that the setup for buildroot was correct before actually building the new package. And to test the build of the new package I need to rebuild everything else first (or most everything else). So the rootfs build is running now.
After this test completes and any fixes applied I'll check it in. Then I can move to integrating libpibox to the core development build (including making pibox-network-config dependent on it) and then migrating utils/log modules from pibox-network-config to libpibox and making sure everything builds appropriately (and runs, of course).
After that I can migrate piboxd to use libpibox and libpiboxnet. That will be nearly as much work as separating the library from pibox-network-config (which was a lot). I may need to merge some support from piboxd into libpiboxnet.
Finally, I can add new support for config.txt file handling to libpibox and SMB and Bluetooth support to libpiboxnet.
There is light at the end of this tunnel but it's dim and far away. In the end, though, I'll have the start of an API that should make it easier for app development.
#13
Updated by Hammel about 2 years ago
A proper construction of the pibox library has been implemented. Now it needs to be integrated into the dev platform build, which means a new package created for it, similar to the one for pnc.
#14
Updated by Hammel about 2 years ago
libpibox is now integrated into the dev platform. I cross compiled bui-network-config with it and copied it to a Pi to run ldd. Sure enough it is missing the libpibox library (which isn't installed yet on that Pi). So the builds are working as expected.
I can complete porting of the utils module in the library and then remove log.c and utils.c from pnc.
Then I can do the port to libpibox and libpiboxnet to piboxd.
#15
Updated by Hammel almost 2 years ago
In migrating piboxd to libpiboxnet I realized that all pncLoadxxx() funds need to clear their GSLists before they recreate them. That is because piboxd does atomic operations on them (load, update, save) while pnc loads and holds the structures until exit, saving as needed. Switching to clearing the GSList at the start of every load function won't break pnc and satisfies piboxd's use case.
#16
Updated by Hammel almost 2 years ago
Switching to the libraries has completely broken pibox-network-config. It segfaults with no real info on why, other than to suggest it was a problem in gtk or gobject, which I doubt.
The only thing I can do it to put in a bunch of printfs/pncLogger calls to trace where it's dying. I believe I have a memory overwrite problem, but I have to figure out where that happens.
It would probably help to be able to build for my desktop (x86_64) and install locally to test so I could run things like valgrind on it. I think this is what I should do first.
#17
Updated by Hammel almost 2 years ago
Okay, that was easy: autoconf run manually lets me build and install libpibox and pibox-network-config on the localhost. So that will allow me to test it easier.
Now to see if I can run it with gdb or valgrind to find the segfault.
#18
Updated by Hammel almost 2 years ago
- % Done changed from 20 to 30
Finally ported pnc library and updated pibox-network-config so it works correctly with it. I tested it on my desktop and on the target and all seems to be working. All changes commited and pushed upstream.
Now I can return to porting piboxd.
#19
Updated by Hammel almost 2 years ago
- Priority changed from Urgent to Immediate
#20
Updated by Hammel over 1 year ago
- Severity changed from 02 - High to 01 - Critical
#21
Updated by Hammel over 1 year ago
- % Done changed from 30 to 50
Piboxd appears to be fully ported now. However, I need to run all the unit tests to be certain.
I also need to check in the changes to the pibox main repo that updates to use pibox-network-config from git instead of an archive.
Once these are done I need to migrate new feature libraries to new issues instead of tracking them here. This issue was just for porting to the use of the shared libraries (libpibox and pibox-network-config).
#22
Updated by Hammel over 1 year ago
- % Done changed from 50 to 60
Found a number of compile time issues in the pnc and pibox libraries that arose from upgrading to Fedora 22 and gcc 5.1 native on intel for testing. Those have all been fixed and and pushed upstream.
I've updated the test script to run the server automatically and shut it down when tests exit. I run it with valgrind to make it easier to find where a segfault occurs, if any. That's easier than doing it with gdb.
Right now I'm failing on test "sipv4" where the 2nd DNS address is not being added to the resolv.conf file. It looks like the request is valid but the saving does not save all three values. Look for pncSetResolvField() in pnc/save.c and add additional debug to make sure I can tell what was passed to the library. I should also add some code to piboxd/net.c to make sure the request is actually correct (are all three values being passed correctly?).
#23
Updated by Hammel over 1 year ago
- % Done changed from 60 to 80
Completed port of piboxd to new libraries. Had to fix a number of bugs in the libraries that were exposed by the unit tests for piboxd. All changes tested, committed and pushed upstream.
Now to create new issues for the new features to be added to the new libraries.
#24
Updated by Hammel over 1 year ago
- Status changed from In Progress to Closed
- % Done changed from 80 to 100
Done. The smb support will not be ported because it's not really about configuration files. It's more about mounting/unmounting. And that should probably stay in piboxd for now.
Closing issue.