Action Item #457

Create backend libraries

Added by Hammel almost 2 years ago. Updated over 1 year ago.

Status:ClosedStart date:12 Jul 2015
Priority:ImmediateDue date:
Assignee:Hammel% Done:

100%

Category:-
Target version:PiBox - 0.11.0
Severity:01 - Critical

Description

This is the start of the API layer in PiBox.

Libraries to create:
  1. Network (from piboxd, bui-network-config)
  2. Include smb support
  3. Add bluetooth config
  4. config.txt handler - changing RPi configuration such as audio output and hdmi setup
  5. App messaging (see RM #433)

More to come.

See

Associated revisions

Revision 575f47cf
Added by Hammel almost 2 years ago

RM #457: First step in creating a library for bui-network-config: clean up the build so it's generated the same as
all other PiBox packages and create an opkg for it (so it's easier to update).

Revision 575f47cf
Added by Hammel almost 2 years ago

RM #457: First step in creating a library for bui-network-config: clean up the build so it's generated the same as
all other PiBox packages and create an opkg for it (so it's easier to update).

Revision 30def81b
Added by Hammel almost 2 years ago

RM #457: Quiet complaints from autoconf on build. Cleanup generated files on distclean.

Revision 30def81b
Added by Hammel almost 2 years ago

RM #457: Quiet complaints from autoconf on build. Cleanup generated files on distclean.

Revision 969dce89
Added by Hammel almost 2 years ago

RM #457: Integrate libtool use.

Revision 969dce89
Added by Hammel almost 2 years ago

RM #457: Integrate libtool use.

Revision 1e76d8af
Added by Hammel almost 2 years ago

RM #457: Integrate libtool use to generate shared library for load/save functions.

Revision 1e76d8af
Added by Hammel almost 2 years ago

RM #457: Integrate libtool use to generate shared library for load/save functions.

Revision 79d7c00f
Added by Hammel almost 2 years ago

RM #457: Make sure all installed files are properly owned.

Revision 79d7c00f
Added by Hammel almost 2 years ago

RM #457: Make sure all installed files are properly owned.

Revision 547983a0
Added by Hammel almost 2 years ago

RM #457: Set include install directory to buinet. Make sure cross tools are available to install processing.

Revision 547983a0
Added by Hammel almost 2 years ago

RM #457: Set include install directory to buinet. Make sure cross tools are available to install processing.

Revision a61dbd2e
Added by Hammel almost 2 years ago

RM #457: Change package name to buinet so headers get put into a decent directory.

Revision a61dbd2e
Added by Hammel almost 2 years ago

RM #457: Change package name to buinet so headers get put into a decent directory.

Revision 58b3d191
Added by Hammel almost 2 years ago

RM #457: Converted from bui- to pibox- and changed the library prefix to pnc.

Revision 58b3d191
Added by Hammel almost 2 years ago

RM #457: Converted from bui- to pibox- and changed the library prefix to pnc.

Revision de4efbab
Added by Hammel almost 2 years ago

RM #457: Add --version-info to library naming.

Revision de4efbab
Added by Hammel almost 2 years ago

RM #457: Add --version-info to library naming.

Revision 5d819f71
Added by Hammel almost 2 years ago

RM #457: Change all references to bui-network-config to pibox-network-config.

Revision 5d819f71
Added by Hammel almost 2 years ago

RM #457: Change all references to bui-network-config to pibox-network-config.

Revision 63e3fdc5
Added by Hammel almost 2 years ago

RM #457: Fix header directory name for library in packaging.

Revision 63e3fdc5
Added by Hammel almost 2 years ago

RM #457: Fix header directory name for library in packaging.

Revision 900731f4
Added by Hammel almost 2 years ago

RM #457: More migration to a proper library API: merged save.h and load.h to pnc.h (public API) and pncPrivate.h (internal API) and migrated all data structure access in those modules to get/set functions. Use libtools features to export only regex specified functions.

Revision 900731f4
Added by Hammel almost 2 years ago

RM #457: More migration to a proper library API: merged save.h and load.h to pnc.h (public API) and pncPrivate.h (internal API) and migrated all data structure access in those modules to get/set functions. Use libtools features to export only regex specified functions.

Revision 27351716
Added by Hammel almost 2 years ago

RM #457: Renamed upstream project to pibox/pibox-network-config and adjusted cdtools accordingly.

Revision 27351716
Added by Hammel almost 2 years ago

RM #457: Renamed upstream project to pibox/pibox-network-config and adjusted cdtools accordingly.

Revision 582592a3
Added by Hammel almost 2 years ago

RM #457: Updated bui-network-config package to build with library from cleaned up upstream repo.

Revision 520b73a7
Added by Hammel almost 2 years ago

RM #457: Migrate to pibox-network-config from bui-network-config.

Revision 3d9b9eff
Added by Hammel almost 2 years ago

RM #457: Convert to a properly built library package, with proper library API structure. This is an initial migration from app-specific functions such as logging and utilities to a common library.

Revision a6966526
Added by Hammel almost 2 years ago

RM #457: Fix deprecated reference to loggerLocal to use new library naming prefix.

Revision b85660fe
Added by Hammel almost 2 years ago

RM #457: Ported to use libpibox's logger service.

Revision b85660fe
Added by Hammel almost 2 years ago

RM #457: Ported to use libpibox's logger service.

Revision 60358342
Added by Hammel almost 2 years ago

RM #457: Added libpibox to development platform build.

Revision 838fa930
Added by Hammel almost 2 years ago

RM #457: Add setter/getter for log verbosity level.

Revision 468a9a06
Added by Hammel almost 2 years ago

RM #457: Corrected call to get verbosity API function in DBG macros.

Revision 07a382d8
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.

Revision 07a382d8
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.

Revision a48d8c6d
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.

Revision a48d8c6d
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup. Moved showIPInStatusBar to callbacks.c and out of network library.

Revision f2747311
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup.

Revision f2747311
Added by Hammel almost 2 years ago

RM #457: Switched to using g_strdup from strdup.

Revision f26016db
Added by Hammel almost 2 years ago

RM #457: Mirgrate stripNewline() calls to use libpibox pncStripNewline().

Revision f26016db
Added by Hammel almost 2 years ago

RM #457: Mirgrate stripNewline() calls to use libpibox pncStripNewline().

Revision 5fc10641
Added by Hammel almost 2 years ago

RM #457: Remove all references to local utils/log modules in favor of libpibox versions.
Changed pncUpdateInterface() to take addr, netmask and gateway arguments instead of calling back to the UI to get them. Caller now makes that UI call before calling pncUpdateInterface.
Removed UI callbacks from network library. Callers now handle UI callbacks before calling library API functions.

Revision 5fc10641
Added by Hammel almost 2 years ago

RM #457: Remove all references to local utils/log modules in favor of libpibox versions.
Changed pncUpdateInterface() to take addr, netmask and gateway arguments instead of calling back to the UI to get them. Caller now makes that UI call before calling pncUpdateInterface.
Removed UI callbacks from network library. Callers now handle UI callbacks before calling library API functions.

Revision afc9d162
Added by Hammel almost 2 years ago

RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.

Revision afc9d162
Added by Hammel almost 2 years ago

RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.

Revision 2c7452cd
Added by Hammel almost 2 years ago

RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.

Revision 2c7452cd
Added by Hammel almost 2 years ago

RM #457: Remove UI callbacks and update API to support passing in required character strings for address type.

Revision ce8d51d3
Added by Hammel almost 2 years ago

RM #457: Remove gdk dependency from network library.

Revision ce8d51d3
Added by Hammel almost 2 years ago

RM #457: Remove gdk dependency from network library.

Revision 3da559da
Added by Hammel almost 2 years ago

RM #457: Initial porting to libpibox and libpiboxnet. Still much work to be done in the port to libpiboxnet.

Revision 9df9df58
Added by Hammel almost 2 years ago

RM #457: Added pncSetInterfaceField() to libpiboxnet API.

Revision 9df9df58
Added by Hammel almost 2 years ago

RM #457: Added pncSetInterfaceField() to libpiboxnet API.

Revision 1b99588e
Added by Hammel almost 2 years ago

RM #457: Add missing clear and remove functions to API needed by piboxd.

Revision 1b99588e
Added by Hammel almost 2 years ago

RM #457: Add missing clear and remove functions to API needed by piboxd.

Revision 04251634
Added by Hammel almost 2 years ago

RM #457: Add missing reference to glib.h in pnc.h so library users don't need to.

Revision 04251634
Added by Hammel almost 2 years ago

RM #457: Add missing reference to glib.h in pnc.h so library users don't need to.

Revision 67c9bf93
Added by Hammel almost 2 years ago

RM #457: Remove return value from pncSetInterfaceField. Check for invalid idx in get/setHostAPField().

Revision 67c9bf93
Added by Hammel almost 2 years ago

RM #457: Remove return value from pncSetInterfaceField. Check for invalid idx in get/setHostAPField().

Revision 156a9b8c
Added by Hammel almost 2 years ago

RM #457: Fix prototype for pncLoggerInit() to include gchar pointer parameter.

Revision b2923164
Added by Hammel almost 2 years ago

RM #457: Pass log file name (if any) to pncLoggerInit().

Revision b2923164
Added by Hammel almost 2 years ago

RM #457: Pass log file name (if any) to pncLoggerInit().

Revision 6c049d75
Added by Hammel almost 2 years ago

RM #457: Change from gchar to char in prototype for pncLoggerInit to match function.

Revision 0e7a4470
Added by Hammel almost 2 years ago

RM #457: Fixed leaks exposed by Valgrind. Fixed errors in startup not exposed before switch to library that have to do with using uninitialized variables in UI. Switched to use static library for pibox-network-config, possibly only for initial testing.

Revision 0e7a4470
Added by Hammel almost 2 years ago

RM #457: Fixed leaks exposed by Valgrind. Fixed errors in startup not exposed before switch to library that have to do with using uninitialized variables in UI. Switched to use static library for pibox-network-config, possibly only for initial testing.

Revision a2d0db26
Added by Hammel almost 2 years ago

RM #457: There were many mistakes in the port to the pnc library. These updates fix all known errors and allow pibox-network-config to properly generate data files under test mode. Removed glade files - this is no longer managed by glade. If you want UI updates, you gotta do it by hand now.

Revision a2d0db26
Added by Hammel almost 2 years ago

RM #457: There were many mistakes in the port to the pnc library. These updates fix all known errors and allow pibox-network-config to properly generate data files under test mode. Removed glade files - this is no longer managed by glade. If you want UI updates, you gotta do it by hand now.

Revision 67a1ebcb
Added by Hammel almost 2 years ago

RM #457: Add debug symbols to library to make it easier to debug with gdb.

Revision 337903c9
Added by Hammel almost 2 years ago

RM #457: Add pncToLower() function.

Revision a2a32190
Added by Hammel almost 2 years ago

RM #457: Revert to shared library use for pibox-network-config from static library used in previous debugging.

Revision a2a32190
Added by Hammel almost 2 years ago

RM #457: Revert to shared library use for pibox-network-config from static library used in previous debugging.

Revision 6215bffe
Added by Hammel over 1 year ago

RM #457: Migrate from archive download to git download for pibox-network-config repo.

Revision 38a2c9c0
Added by Hammel over 1 year ago

RM #457: Cleanup compile time warnings and errors introduced when upgrading to gcc 5.1.

Revision 38a2c9c0
Added by Hammel over 1 year ago

RM #457: Cleanup compile time warnings and errors introduced when upgrading to gcc 5.1.

Revision 807dde60
Added by Hammel over 1 year ago

RM #457: Add check for null argument to pncNetInterfaceValid().

Revision 807dde60
Added by Hammel over 1 year ago

RM #457: Add check for null argument to pncNetInterfaceValid().

Revision 63d24ab2
Added by Hammel over 1 year ago

RM #457: Add missing prototype for pncStripNewline to utils.h. Add missing header reference for ctype.h to utilc.c.

Revision d7cb6f7d
Added by Hammel over 1 year ago

RM #457: Prevent segfaults if pncLogger is called with null format string.

Revision 1b53a723
Added by Hammel over 1 year ago

RM #457: Fix potential segfaults by testing input arguments to callbacks for list handlers.

Revision 1b53a723
Added by Hammel over 1 year ago

RM #457: Fix potential segfaults by testing input arguments to callbacks for list handlers.

Revision cd206848
Added by Hammel over 1 year ago

RM #457: Remove extraneous file.

Revision 35ed649d
Added by Hammel over 1 year ago

RM #457: Extend suppression file to ignore gobject, libpixman and a few others that are not part of the code under test.

Revision bf5858d1
Added by Hammel over 1 year ago

RM #457: Extend test to support running with valgrind, split out SIPV4 subtests to be callable individually and report pass/fail, launch piboxd and kill it automatically when running the tests. All of this is part of the port of piboxd to libpiboxnet.

Revision 3a2ef30b
Added by Hammel over 1 year ago

RM #457: Completed port for libpiboxnet by applying fixes (mostly in the new libraries: libpiboxnet and libpixbox) and removing old code that is now in the libraries. All functionality passes the server tests.

History

#1 Updated by Hammel almost 2 years ago

  • Project changed from PiBox to pibox-network-config
  • Category deleted (04 - Applications)

#2 Updated by Hammel almost 2 years ago

  • Subject changed from Move bui-network-config backend into a library to Create backend libraries
  • Description updated (diff)

#3 Updated by Hammel almost 2 years ago

  • Description updated (diff)

#4 Updated by Hammel almost 2 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

Restructured bui-network-config's build so it's just like all other PiBox builds (using autoreconf -i to gen the configure script). Also added opkg packaging to it. This will make it easier to test the migration to a library + frontend from the stand alone app.

Now I need to verify that the core builds this correctly.

#5 Updated by Hammel almost 2 years ago

  • % Done changed from 10 to 20

bui-network-config build verified working with existing structure of pibox dev platform build system. No changes required.

Now to break bui-network-config into a core and a shared library.

#6 Updated by Hammel almost 2 years ago

  • Description updated (diff)

#7 Updated by Hammel almost 2 years ago

I've created a shared lib and the local package-build works fine. Everything gets installed where it should (/usr/include/buinet for headers, /usr/lib for libs). But when I run it from Buildroot it doesn't complete correctly. The libs and program get installed fine. The headers are missing. The log says they got installed, but they aren't there. No idea why yet.

#8 Updated by Hammel almost 2 years ago

The command to install the headers works if I run it manually. So maybe there is something weird about rerunning the buildroot build? Maybe I need to rerun the whole thing. Yuck. I shouldn't have to do that.

I'll rerun the build tonight from scratch, just to see if that has any affect.

#9 Updated by Hammel almost 2 years ago

  • Description updated (diff)

#10 Updated by Hammel almost 2 years ago

  • Description updated (diff)

#11 Updated by Hammel almost 2 years ago

I've changed the name of the program to pibox-network-config and the prefix to pnc for all library routines. This builds fine. I'll commit and push.

Then I need to create an init.c that will be part of the library. It's purpose is to abstract the filenames and provide get/set functions for them. I'll also need to make INTERFACE_T abstract so its members can only be retrieved via a get function (via GList's).

  • Once the init module is written then I can implement CLI handling for specifying the filenames when testing.
  • The bui-network-config package will need to change to pibox-network-config.
  • The upstream repo should be changed from bui-network-config to pibox-network-config

utils.c and log.c need to be merged into libPibox. That means libPibox will need to be part of the development platform build.

#12 Updated by Hammel almost 2 years ago

bui-network-config has now been changed to pibox-network-config in the source tree and in the upstream repo. The repo was also moved out from the xbui parent to the pibox parent. The source tree now builds the pibox-network-config binary along with a shared library and public API headers. I removed all direct references to private structures in the library from the application and switched to using only getter/setter methods. Everything is tested on the target and seems to be working correctly. I've even added opkg support to the build so future updates don't require rebuilding the rootfs.

So now I'm updating the core development platform to use the updated package and repo names. This will take a while because I had to wipe the existing rootfs build to properly verify that the setup for buildroot was correct before actually building the new package. And to test the build of the new package I need to rebuild everything else first (or most everything else). So the rootfs build is running now.

After this test completes and any fixes applied I'll check it in. Then I can move to integrating libpibox to the core development build (including making pibox-network-config dependent on it) and then migrating utils/log modules from pibox-network-config to libpibox and making sure everything builds appropriately (and runs, of course).

After that I can migrate piboxd to use libpibox and libpiboxnet. That will be nearly as much work as separating the library from pibox-network-config (which was a lot). I may need to merge some support from piboxd into libpiboxnet.

Finally, I can add new support for config.txt file handling to libpibox and SMB and Bluetooth support to libpiboxnet.

There is light at the end of this tunnel but it's dim and far away. In the end, though, I'll have the start of an API that should make it easier for app development.

#13 Updated by Hammel almost 2 years ago

A proper construction of the pibox library has been implemented. Now it needs to be integrated into the dev platform build, which means a new package created for it, similar to the one for pnc.

#14 Updated by Hammel almost 2 years ago

libpibox is now integrated into the dev platform. I cross compiled bui-network-config with it and copied it to a Pi to run ldd. Sure enough it is missing the libpibox library (which isn't installed yet on that Pi). So the builds are working as expected.

I can complete porting of the utils module in the library and then remove log.c and utils.c from pnc.

Then I can do the port to libpibox and libpiboxnet to piboxd.

#15 Updated by Hammel almost 2 years ago

In migrating piboxd to libpiboxnet I realized that all pncLoadxxx() funds need to clear their GSLists before they recreate them. That is because piboxd does atomic operations on them (load, update, save) while pnc loads and holds the structures until exit, saving as needed. Switching to clearing the GSList at the start of every load function won't break pnc and satisfies piboxd's use case.

#16 Updated by Hammel almost 2 years ago

Switching to the libraries has completely broken pibox-network-config. It segfaults with no real info on why, other than to suggest it was a problem in gtk or gobject, which I doubt.

The only thing I can do it to put in a bunch of printfs/pncLogger calls to trace where it's dying. I believe I have a memory overwrite problem, but I have to figure out where that happens.

It would probably help to be able to build for my desktop (x86_64) and install locally to test so I could run things like valgrind on it. I think this is what I should do first.

#17 Updated by Hammel almost 2 years ago

Okay, that was easy: autoconf run manually lets me build and install libpibox and pibox-network-config on the localhost. So that will allow me to test it easier.

Now to see if I can run it with gdb or valgrind to find the segfault.

#18 Updated by Hammel almost 2 years ago

  • % Done changed from 20 to 30

Finally ported pnc library and updated pibox-network-config so it works correctly with it. I tested it on my desktop and on the target and all seems to be working. All changes commited and pushed upstream.

Now I can return to porting piboxd.

#19 Updated by Hammel almost 2 years ago

  • Priority changed from Urgent to Immediate

#20 Updated by Hammel over 1 year ago

  • Severity changed from 02 - High to 01 - Critical

#21 Updated by Hammel over 1 year ago

  • % Done changed from 30 to 50

Piboxd appears to be fully ported now. However, I need to run all the unit tests to be certain.

I also need to check in the changes to the pibox main repo that updates to use pibox-network-config from git instead of an archive.

Once these are done I need to migrate new feature libraries to new issues instead of tracking them here. This issue was just for porting to the use of the shared libraries (libpibox and pibox-network-config).

#22 Updated by Hammel over 1 year ago

  • % Done changed from 50 to 60

Found a number of compile time issues in the pnc and pibox libraries that arose from upgrading to Fedora 22 and gcc 5.1 native on intel for testing. Those have all been fixed and and pushed upstream.

I've updated the test script to run the server automatically and shut it down when tests exit. I run it with valgrind to make it easier to find where a segfault occurs, if any. That's easier than doing it with gdb.

Right now I'm failing on test "sipv4" where the 2nd DNS address is not being added to the resolv.conf file. It looks like the request is valid but the saving does not save all three values. Look for pncSetResolvField() in pnc/save.c and add additional debug to make sure I can tell what was passed to the library. I should also add some code to piboxd/net.c to make sure the request is actually correct (are all three values being passed correctly?).

#23 Updated by Hammel over 1 year ago

  • % Done changed from 60 to 80

Completed port of piboxd to new libraries. Had to fix a number of bugs in the libraries that were exposed by the unit tests for piboxd. All changes tested, committed and pushed upstream.

Now to create new issues for the new features to be added to the new libraries.

#24 Updated by Hammel over 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 80 to 100

Done. The smb support will not be ported because it's not really about configuration files. It's more about mounting/unmounting. And that should probably stay in piboxd for now.

Closing issue.

Also available in: Atom PDF