Friday, October 30, 2020

Firefox and Your Bookmarks

 Recovering your bookmarks from firefox

I am writing this as an alternate method of retrieving your bookmarks, other than using Firefox itself.  I had an issue where the main drive in one of my servers decided it didn't want to boot anymore.  I could still access the data on it via an external cable, but the system became unbootable.  

In searching, I found that Firefox stores its book marks in a file called 'places.sqlite'.  That's correct, it is an sqlite database, which means bookmarks are not the only things in this file.  The file is stored in ~/.mozilla/firefox/<blah>.default-release.  Now, i am on Firefox 82 and my full path is:  ~/.mozilla/firefox/6pcydzru.default-release.  

Now, the first thing to do is to go to that directory and copy that file to some place else (don't move it, copy it).

    cp places.sqlite ~   <- Yes, I simply copied it to my home directory

The reason for this is that Firefox locks this file and working around that lock is as simple as copying it to another location.  And you can keep the same name, if you wish, its not an issue.

Now, if you were to load up that file into sqlite3 and take a look, you would see that the bookmarks are certainly there, in a table called moz_bookmarks, but the urls are hashed and not readable.  Here is what it looks like when I loaded it up and took a look:

$ sqlite3 places.sqlite
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
moz_anno_attributes    moz_historyvisits      moz_meta             
moz_annos              moz_inputhistory       moz_origins          
moz_bookmarks          moz_items_annos        moz_places           
moz_bookmarks_deleted  moz_keywords         
sqlite> select * from moz_bookmarks;
7|2||2|0|Mozilla Firefox|||1602647046960000|1602647046960000|hk7fo0BVoSNm|0|1
8|1|3|7|0|Help and Tutorials|||1602647046960000|1602647046960000|mtMmQ-E9oI58|0|1
9|1|4|7|1|Customize Firefox|||1602647046960000|1602647046960000|DrTu2wYZmIj-|0|1
10|1|5|7|2|Get Involved|||1602647046960000|1602647046960000|-sWrRJ5NeO8R|0|1
11|1|6|7|3|About Us|||1602647046960000|1602647046960000|LCI5sKvqimve|0|1
12|2||2|1|Ubuntu and Free Software links|||1602647046960000|1602647046960000|wvi_2QDOZLm9|0|1
14|1|8|12|1|Ubuntu Wiki (community-edited website)|||1602647046960000|1602647046960000|6A8e4nK1328m|0|1

This isn't really usable, by any stretch, at the moment.  But a little googling, and I found a site called, whom posted a nice shell script which would do the job for you.  
The link to this is:

The script is on there for you to copy to your system, I am not going to repost it here.  But, for readability, I will say that I modified a single line in the script.  Here is the modified line:

echo "$3  <A HREF=\"$url\">$title</A><br />"

As you can see, I added a '<br />' so each link would be on its own line, making things much nicer, and one entry per line.

Once you have the script in place (and the modification, if you so choose), you can run it like this: 

 ./ ./places.sqlite > bookmarks.html

You will need to ensure that you replace the './' with the correct path to the files, if that is incorrect.  For me, I had them in the same directory

This doesn't seem huge thing, and sure, you could use the gui, but that is not me.  I prefer to do things on the command line and figure them out. Besides, I had fun doing it and learning about it, and thought I would share with everyone in case anyone else found it fun and fascinating.


Thursday, February 28, 2019

Untracking files in git

In one of my current projects, pyblueprint-py3, I have a config file.  This file gets edited locally after cloning the repository, and those changes get seen by git.  That is a problem.  This is a file that is distributed with default values that the user edits, and git does not need to know about these changes.  But, how do we tell git not to care about changes to a file?  Good question, and here is the answer.

For your config file (or whatever file it is that you want to prevent git from tracking), create it in your repository, and then issue the following command:

    git update-index --skip-worktree file_name

The file_name should include the path if it isn't in the current directory (ie: its in a subdirectory).

If the file that you don't want to track is already on git's radar, then you can do the following:

    git rm 
    git rm -r --cached 
    git commit -m'- Cleaning up and removing file to be ignored'
    git push

Then, add the file and edit the conf file with the generic placeholder and do this:

    git add 
    git update-index --skip-worktree 
    git commit -m'- Committing new, untracked file'
    git push

One thing to remember, is that when someone clones your repository, before they do anything, they will need to reissue the 'git update-index' command above for the specific file you untracked.  That should be noted in your documentation.

pyblueprint-py3 Now Exists

What a crazy 3/4 of a year its been.  The last company I worked for kept going down hill for the employees with a ton of management change-over, which was in no way good.  Eventually, things had to change and I needed to get back to a happy place.  At the turn of the year I started a new job, and BOY, am I happier.  While I am still ramping up at my new place of employment, I am certainly having a lot more fun here than I did the last couple years at my old job.  Considering what is going on over there, I do not envy my old colleagues.

Enough about the craziness.  I recently sat down and took the time to convert the pyblueprint project over to using Python3.  Not only that, there is a new option for specifying a language.  It defaults to python, but it will create a directory structure for whatever language you choose.  But, python and Ruby have support for the virtualenviroment that this project sets up (as does Go and NodeJS, but I haven't yet put together docs and such for that.  That is coming though.  I migrated the project to a new name, call pyblueprint-py3.  Feel free to download it and play with it.  Hopefully it works for you.  If there are issues, please open a ticket in gitlab under the project.  Also, I am open to feature requests, should anyone have any. 

Hopefully I can get more additions on to this project in the near future. 

Friday, July 06, 2018

A new home for my projects and a project update

New Home

I know that there is always an expectation that, if you are a developer of some sort, you store your code into Github (at least that is the first one that comes to people's mind).  I have a Github acct, but, because their private repos are not free, I haven't truly used them for storing my repos.  And considering that Microsoft now owns them, I really won't be using them for my code.

Up until now, I have relied on Bitbucket (currently owned by Atlassian), as it offers unlimited free private and public repos.  We have both Bitbucket and Gitlab at my work (local installs for both).  After all of the increased fanfare for Gitlab (after the Microsoft purchase of Github) and my exposuer to it at work, I have made the switch to store my repos there as well.  Unless they change their policies, I will probably be with them quite a while.  

Project Update:

I did want to say that I have migrated my 'pyblueprint' project over to Gitlab.  Thankfully Gitlab makes the migration to them painless, and it was just a click of a couple of buttons to get the repo moved over.  The 'hard part', if you will (although not really), was fixing my git remote to point to the new repository location.  

I do want to say that I have added support to the project for creating a repository up in Gitlab.  Seeing as how I am now using it, why not.  :)  While testing out the application to ensure everything was still working afterwards, I discovered that my checks were not working.  Why?  Because the maintainers of 'pip' decided that, as of pip 10.0.0, they no longer support pip bing used programatically.  If you drop into IDLE, do an 'import pip' and then do a 'dir(pip)', you will now notice that there are no methods associated (aside from the underscore methods).  This completely broke my checks as I had relied on that useful bit.  No worries though, the checks are fixed and functioning as expected.  

Also, if anyone is using this application (don't know if anyone is) and they would like support for an alternative code repository, please feel free to open an enhancement ticket and I will see about adding support for it.  

That's it for now, happy coding!

Saturday, June 30, 2018

Setting up MELPA for Emacs packages

In my journey to learn emacs, one of the first things that I wanted to do was change the theme so that it was more pleasing to the eye.  Anyone who uses a tool enough knows that if you can personalize it for you, you personalize it. 

So in researching themes ( see the emacs themes site ), I kept seeing references to MELPA, which is an emacs package manager.  Sweet!  Unfortunately, my quest to install the theme I wanted was met with the fact that it is NOT configured to use MELPA by default.  Thus, the reason for this write-up. 

In order to configure emacs to see the MELPA packages, you need to edit your .emacs file in your home directory.  If you have at least opened emacs, this file should exist.  If not, just create it and open it for editing.

To correctly enable MELPA, you will need the following as the first section of the file:

;; load emacs 24's package system. Add MELPA repository.
(when (>= emacs-major-version 24)
  (require 'package)
   ;; '("melpa" . "") ; many packages won't show if using stable
   '("melpa" . "")
 After you save the .emacs file, you will need to restart emacs if it is already running.  If not, start it up.  Once running, you will need to his what in the emacs world is referred to as M-x.  On a mac, that is Option + the 'x' key.  I don't run Windows, so I really couldn't begin to tell you what it is on there. 

That will drop you into the M-x mini buffer.  There, type 'list-packages'.  In the main window, a columned list should appear.  The 4th column is 'archive' and should have packages that say 'melpa' (as well as 'gnu').  If so, you are good to install the theme you found. 

To install a melpa package, drop back into the M-x mini buffer and type 'package-install'.  This will change the prompt to "Install package: " which will wait for a package name.  The nice thing is that tab-complete works in emacs.  So start typing a package name and hit tab, and a list will appear in the are right above the mini buffer. Once you complete installing the packages you need, go ahead and restart emacs. 

I certainly hope this makes your emacs package installation easier.  :)

Straddling the line between editors

Since I started in Unix/Linux about 20 years ago, I have been a staunch user and supporter of vi.  I have known plenty of people that were emacs aficionados, but haven't really taken the time to learn it to the point of being proficient.  I did look at it a few years ago after starting my current job, but with all of the new things I was learning, I forwent it in favor of the things I actually needed to learn at my job.

Now, 5 1/2 year later, I have decided that I am at a point where I finally have time to learn it.  I know what vi lovers are thinking, "hell no!", and I get that.  I was like that for years.  But, after getting to work for the last 5 1/2 years with a friend that I have now known for 16 years, I have seen him operate pretty seemlessly in emacs, without needing to go to the shell that often.  I have seen some pretty cool features that nicely streamlined his workflow.

Its never easy picking up a new tool, that's for sure, especially when you like the features of your current tool, but sometimes you have to make concessions to go farther.  All that said, don't be surprised if you see emacs related posts here, as I am wanting to share what I am learning.

In closing I will provide the two initial links that I am using to start my journey down the emacs path:

Absolute Beginner's Guide To Emacs
Practical Emacs Tutorial

Hopefully, if you decide to also venture down this road, those will help you as well.  I will say that so far, the Absolute Beginner's Guide has been awesome!!

Saturday, May 05, 2018

Setup WiFi to auto-connect regardless of logged in state (UPDATED)

There are plenty of times where I would like to start up my desktop machine (running Ubuntu 16.04 LTS) and just have it boot and me log into it via SSH so that I can get some work done.  Well, in order to achieve that wish, you need to have your wifi connection auto-connnect at boot time.

To do this, there are some things you need to do:

1.  Go to your wifi connection in the upper right of your screen and click on "Edit Connections".
2.  Once in there, you need to click on your connected wifi ssid and select "Edit".
3.  Go to the "General" tab and select "All users may connect to this network".  (without this, the rest of this is moot and it will not work)

Ok, that is the only gui portion of this setup.  Next:

4.  Create the file /etc/wpa_supplicant.conf and add the content that follows:

ctrl_interface=DIR=/var/run/wpa_supplicant group=wheel
In the above code, you will input your ssid that you would like to connect to and also enter your password for the connection.  If you are using an encryption method other than WPA/WPA2, then you will need to reference the man page for wpa_supplicant.conf.  There are plenty of examples to help you out. 

After you have that file all setup, save it.

5.  Issue the following command on the command line, as root or with sudo:

wpa_supplicant -B -i wlp4s0 -c /etc/wpa_supplicant.conf

You should see a message that says something to the effect that wpa_supplicant initialization was successful.

After you have completed those steps, go ahead and reboot the machine and watch the login screen.  If all is correct and right with the configuration, you should see the wifi connection show as connected.  You can test this by ssh'ing to the machine.  Enjoy!

UPDATE:  I took the opportunity to reinstall my Linux system to go from Ubuntu 16.04 LTS to Ubuntu 18.04.  I know its a bit extreme to do a full re-install, but:

1.  An distribution upgrade wasn't yet available
2.  There wasn't anything I couldn't re-setup on the machine
3.  Ubuntu moved away from Unity to the Gnome desktop, so I wanted everything clean and fresh.

I will say this though, auto-connection of the wifi (seeing as how you tell it how to connect during installation) just WORKS with Ubuntu 18.04.  You just have to make sure to install the openssh-server package.

Tuesday, May 01, 2018

How to list available versions of a Pypi package

While working on requirements that someone had for a pypi package, I found myself needing to list out the versions of the package that were available.  I didn't want to have to go to the pypi website, navigate to the package and then figure out how to see the available versions.  So, to the Googles I went.

In order to install a specific version of a package, you would run:

pip install =1.0.0

in order to install version 1.0.0 of said package.  But, if you provide nonsense as the option, like so:

pip install =blah

then what you get back is an error listing all of the available versions for that package:

> pip install mod_wsgi==blah
Collecting mod_wsgi==blah
Could not find a version that satisfies the requirement mod_wsgi==blah (from versions: 4.1.0, 4.1.1, 4.1.2, 4.1.3, 4.2.0, 4.2.1, 4.2.2, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.3.0, 4.3.1, 4.3.2, 4.4.0, 4.4.1, 4.4.2, 4.4.3, 4.4.4, 4.4.5, 4.4.6, 4.4.7, 4.4.8, 4.4.9, 4.4.10, 4.4.11, 4.4.12, 4.4.13, 4.4.14, 4.4.15, 4.4.16, 4.4.17, 4.4.18, 4.4.19, 4.4.20, 4.4.21, 4.4.22, 4.4.23, 4.5.0, 4.5.1, 4.5.2, 4.5.3, 4.5.4, 4.5.5, 4.5.6, 4.5.7, 4.5.8, 4.5.9, 4.5.10, 4.5.11, 4.5.12, 4.5.13, 4.5.14, 4.5.15, 4.5.16, 4.5.17, 4.5.18, 4.5.19, 4.5.20, 4.5.21, 4.5.22, 4.5.23, 4.5.24, 4.6.0, 4.6.1, 4.6.2, 4.6.3, 4.6.4)
No matching distribution found for mod_wsgi==blah

Good to know its that easy.

Sunday, April 29, 2018

python pip throwing SSL: TLSV1 alert (SOLUTION)

I recently ran into an issue with Python pip, where I attempted to run a 'pip install' of a module and was presented with the following error:
> pip install pymongo
Collecting pymongo
Could not fetch URL There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) - skipping
Could not find a version that satisfies the requirement pymongo (from versions: )
No matching distribution found for pymongo

No matter what I did (install, update and even just search the pip repository) I was presented with this same error.  It was certainly a hard, fast stop to whatever I was attempting to do.

But, after some googling and reading of many people's accounts of their trials and tribulations attempting to solve it, I finally found a solution that works.  I happened upon the this posting on StackOverflow, which leads you to the fact that you have to upgrade pip.  Unfortunately, attempting to upgrade it with itself will fail as above.  But, you should instead run the following:

curl | python

That upgraded pip directly from the website using its install script.  It is a work around, yes, but one that seems to have worked for me (and apparently others).  Hopefully this solves your issue(s) as well.

**UPDATE: You will need to run the curl command in each virtualenv environment that you have, as each one will have its own installation of python.

Saturday, December 16, 2017

'_remove_dead_weakref' error after upgrading to Ubuntu 17.10

Well, after finally getting my Flask app running and returning data, I upgraded my system to Ubuntu 17.10 (from 17.04).  Today, I tried to start my flask app and was greeted with a bunch of Python error output and the following error:

ImportError: cannot import name '_remove_dead_weakref'

I was baffled as the app worked the day before.  So after some digging around the Googles, I found that the issue was caused by my upgrade to 17.10.

The fix....... delete your virtual environment and rebuild it.  I am not saying delete your project directory, I am saying delete your envs directory (which is where the executables for the virtual environment are.  Some people put it in the same directory.  If you are like me, you have a central location where you keep them. 

You will need to remove that entire directory, rebuild it and then re-run the virtualenv command to create the virtualized environment and then install any modules that you had previously installed (as they get stored in the directory structure you just deleted.  After that, your app and its parts should work fine.
Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.