Saturday, May 07, 2022

Thoughts On Ubuntu 22.04 (UPDATED)

 So here I am a few days ago, sitting at my computer, working on a project, and a pop-up appears asking me that Ubuntu 22.04 had been release, asking if I wanted to upgrade to it.  (Yes, I know that I can limit what I am notified of, I actually choose to know when the latest is available).  

I think, "Sure, why not." and clicked through the plethora of buttons to get it up upgrade (agreeing to the upgrade no less than 3 or 4 times). So my machine goes into upgrading and I decide to let it do its thing.  An hour or so later, I return and its at a login prompt, as it had rebooted.  Only, the login page is now dark themed.  Very nice!!!  The upgrade results were starting off on a good note.  

While its just a lot of packages that were upgraded, as well as the underlying software, I will say that there was an extremely annoying side-effect of this upgrade.  I have some projects that I have setup, including having done pip installs of packages and such for the software.  It seems that during the upgrade, all of those packages and settings went away.  Seriously?  Just *POOF*?  What the heck, Ubuntu?  Way to blow away my installs and configs.  

So, I had to spend some time re-setting up the stuff I regularly used (and some I was working on).  All of the items, by the way, were already using Python 3, so nothing except a newer version being installed.  

I just wanted to share this annoyance, in case you are planning on upgrading.  You will need to plan on some extra time, re-setting up some things once the upgrade completes.


UPDATE:  After some research, and noting issues seen on other machines (non-ubuntu) and work systems, this seems to be something related to a python3 upgrade.  On my work Mac, it seems that Python3 upgraded, and put itself in a different location, but did not take on what was previously installed.  It caused a ton of confusion in my system.  There were two different versions of pip3 and two different versions of python3.  It took using 'which -a' to figure this out and zero in on the issue.  In the end, I had to use an alias for each on my Mac to set things right.  

Here on my Ubuntu system though, the python3 upgrade simply wiped out everything that was installed with the previous python3 installation, that wasn't part of the standard library.  Thanks for that.  Way to seriously cause broken software that is used constantly.

Friday, April 29, 2022

So Long.... Youtube-DL

 A couple of years ago, I decided to write a quick script called ytdl, short for 'youtube downloader'.  I wrote it as a wrapper for the youtube-dl application, as I was tired of always trying to remember the options that I used when downloading either a video or a song (in audio format only) from youtube. 

Recently, I revisited the application as I needed to grab a couple of things off of youtube, and found that when running it, the downloads were CRAZY slow!  By CRAZY slow, I mean in the area of only getting between 25k and 50k max as a download speed.  I was looking at 1-2 hours to download each of the videos I was wanting to grab.  Considering in the past it would only have taken me a few minutes to download the videos I wanted (in total), I decided to use my Google Foo and see what was up. 

Sure enough, it appears that:

1. youtube.com was throttling the youtube-dl application.  This was confirmed by the plethora of complaints from the community that uses(used) it.  

2. I discovered that youtube-dl was no longer being maintained, and that the last version (from december of last year), was the final version. 

Well, that's not good.  But, what is good, is that there was a replacement...... yt-dlp!!!  Excellent, a successor.  So, I proceeded to download yt-dlp and incorporate it into my script, utterly replacing the old, useless youtube-dl.  

After some tweaking to get the downloaded files the way I like them, I can happily say that I am now seeing download speeds in the megabytes!!!  (7-13Mb, just on the dl's I did today).  

So, I have updated all the download types in the script and pushed up a new version.  If you haven't played with it, please feel free to give it a go.  And, if you have any download sites that you'd like added to the script (that are supported by it), please feel free to open an issue for an improvement.

Here is a link to my ytdl project.  Enjoy!

Wednesday, April 27, 2022

Removing Tag(s) From A Git Repository

 If you work on a team that is constantly developing their code, then you are probably dealing with at least one repository that gets periodically tagged (such as when a code release is performed).  If that is the case, then you have probably (or not) had to deal with setting the tags that get applied to the repository.  

If so, then you would be able to see the tags on a repository with the following command:

   $ git tag

If you are looking for a tag, but you only know part of it, you can certainly use grep to weed through the potentially large list of tags that will be presented to you.  Now, if you happen to set a tag incorrectly, or apply it to the wrong repository (or branch of that repository, for instance), you are not stuck.  You can remove it.  Here are the steps:

  1. First, ensure you are in the repository in question and in the correct branch, then do a 'git pull' to ensure you are at the latest change
  2. Run 'git tag | grep <tag>' to get the tag name you need to remove
  3. Run 'git tag -d <tag_name>'        <--- This will remove the tag in your local repository
  4. Run 'git push --delete origin <tag_name>'    <--- This will remove the tag from the remote repo
  5. Run 'git pull'
From here, you will need to run the 'git tag' command in step 2 and verify that the tag has been removed locally.  You can also then go up to GitHub, Gitlab, etc, and check the repository and ensure that the tag has been removed there as well.   

Wednesday, April 13, 2022

Read Medium Articles For Free

 There is nothing more annoying to someone who is just trying to read about something they are trying to learn about, than to have a site say, "Hey, you've reached your 'free' limit!".  That is exactly what medium.com does with its articles.  You need to log in to read further (so it says), and then, when you do, they STILL won't let you go any further until you pay for the right to read their articles.  Call me cheap, but I believe in the freedom of information.  

So, a little Google-foo and I discovered that there is a workaround for this that works in Chrome/Brave:

- Load up the article you want to read

- In the URL bar at the top, click the little padlock (the one that indicates that the site is secured)

- In there, it will tell you how many cookies there are.  Click on "Cookies".  It will bring up a window similar to this:


- Click on 'medium.com' and click 'Block' down below, and then Done.


Chrome/Brave will then ask you to reload the page.  Once you do, you should be able to read the article without the annoying message telling you that you haven't paid them for the privilege yet.  

Enjoy! 

Sunday, April 10, 2022

Installing Dependencies for dpkg Software Installation

Let's say you download any of the plethora of '.deb' packages that exist out there, and want to install it on your Ubuntu system (or the like).  You would use something similar to the following to do that installation:

    sudo dpkg -i imager_1.7.2_amd64.deb

Now, let's say you run that, but are then presented with output stating that there are a bunch of unmet dependencies.  Disconcerting, sure, but its not the end of the world, for sure.  For example:

$ sudo dpkg -i imager_1.7.2_amd64.deb 

Selecting previously unselected package rpi-imager.

(Reading database ... 195449 files and directories currently installed.)

Preparing to unpack imager_1.7.2_amd64.deb ...

Unpacking rpi-imager (1.7.2) ...

dpkg: dependency problems prevent configuration of rpi-imager:

 rpi-imager depends on libqt5qml5 (>= 5.10.0); however:

  Package libqt5qml5 is not installed.

 rpi-imager depends on qml-module-qtquick2; however:

  Package qml-module-qtquick2 is not installed.

 rpi-imager depends on qml-module-qtquick-controls2; however:

  Package qml-module-qtquick-controls2 is not installed.

 rpi-imager depends on qml-module-qtquick-layouts; however:

  Package qml-module-qtquick-layouts is not installed.

 rpi-imager depends on qml-module-qtquick-templates2; however:

  Package qml-module-qtquick-templates2 is not installed.

 rpi-imager depends on qml-module-qtquick-window2; however:

  Package qml-module-qtquick-window2 is not installed.

 rpi-imager depends on qml-module-qtgraphicaleffects; however:

  Package qml-module-qtgraphicaleffects is not installed.


dpkg: error processing package rpi-imager (--install):

 dependency problems - leaving unconfigured

Processing triggers for mailcap (3.69ubuntu1) ...

Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...

Processing triggers for desktop-file-utils (0.26-1ubuntu2) ...

Processing triggers for hicolor-icon-theme (0.17-2) ...

Errors were encountered while processing:

 rpi-imager


I know that is a lot of output, sorry about that.  But I left it that way for effect and to give a full example.  Now, what you will need to do in this case, is run the following:

        sudo apt -f install -y

That will take and install any dependencies that were found during the previous software installation attempt, and install them.  After that is done, you can re-run the dpkg command to install your software, and it "should" just work and install your software.  ( I say should in quotes, as nothing is guaranteed. )


Thursday, November 04, 2021

PyBlueprint update

 Since I have started this project, I have had an issue where the "checks" option, to verify that everything was correctly installed, has been failing to detect virtualenvwrapper as installed, even though it is in the 'pip list' output.  

After a bunch of digging and research, I have discovered a way to do the checks that seems to be working without issue.  I have installed it on a couple of different systems, both Linux and Mac, and things seem to be functional.  That is a breath of fresh air from me, as I have hated the error happening (and subsequently being ignored by me as I knew it was installed). 

So what is next?  I plan on adding support for more languages (shell, perl, and fix the ruby implementation to be more automated).  I also want to give an option for setting up directories for code that has been checked out from a repo, so that it can be added and setup more easily than trying to remember the commands to add it.  

We shall see how those options come along and hopefully I can advance this software's language support.  If anyone has any requests, by all means, please leave a comment and I will see about adding to my list of items.  Heck, the best way would be to open an enhancement in the repo.  You can do this by creating an issue and title it as an enhancement.  

Happy Geeking!

Friday, October 30, 2020

Firefox and Your Bookmarks

 Recovering your bookmarks from firefox

I am writing this as an alternate method of retrieving your bookmarks, other than using Firefox itself.  I had an issue where the main drive in one of my servers decided it didn't want to boot anymore.  I could still access the data on it via an external cable, but the system became unbootable.  

In searching, I found that Firefox stores its book marks in a file called 'places.sqlite'.  That's correct, it is an sqlite database, which means bookmarks are not the only things in this file.  The file is stored in ~/.mozilla/firefox/<blah>.default-release.  Now, i am on Firefox 82 and my full path is:  ~/.mozilla/firefox/6pcydzru.default-release.  

Now, the first thing to do is to go to that directory and copy that file to some place else (don't move it, copy it).

    cp places.sqlite ~   <- Yes, I simply copied it to my home directory

The reason for this is that Firefox locks this file and working around that lock is as simple as copying it to another location.  And you can keep the same name, if you wish, its not an issue.

Now, if you were to load up that file into sqlite3 and take a look, you would see that the bookmarks are certainly there, in a table called moz_bookmarks, but the urls are hashed and not readable.  Here is what it looks like when I loaded it up and took a look:

$ sqlite3 places.sqlite
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
moz_anno_attributes    moz_historyvisits      moz_meta             
moz_annos              moz_inputhistory       moz_origins          
moz_bookmarks          moz_items_annos        moz_places           
moz_bookmarks_deleted  moz_keywords         
sqlite> select * from moz_bookmarks;
1|2||0|0||||1602647046341000|1604087203227000|root________|1|1
2|2||1|0|menu|||1602647046341000|1602647046960000|menu________|1|3
3|2||1|1|toolbar|||1602647046341000|1602647046988000|toolbar_____|1|3
4|2||1|2|tags|||1602647046341000|1602647046341000|tags________|1|1
5|2||1|3|unfiled|||1602647046341000|1604087203227000|unfiled_____|1|8
6|2||1|4|mobile|||1602647046545000|1602647046943000|mobile______|1|2
7|2||2|0|Mozilla Firefox|||1602647046960000|1602647046960000|hk7fo0BVoSNm|0|1
8|1|3|7|0|Help and Tutorials|||1602647046960000|1602647046960000|mtMmQ-E9oI58|0|1
9|1|4|7|1|Customize Firefox|||1602647046960000|1602647046960000|DrTu2wYZmIj-|0|1
10|1|5|7|2|Get Involved|||1602647046960000|1602647046960000|-sWrRJ5NeO8R|0|1
11|1|6|7|3|About Us|||1602647046960000|1602647046960000|LCI5sKvqimve|0|1
12|2||2|1|Ubuntu and Free Software links|||1602647046960000|1602647046960000|wvi_2QDOZLm9|0|1
13|1|7|12|0|Ubuntu|||1602647046960000|1602647046960000|Zfpq8U7qHTzz|0|1
14|1|8|12|1|Ubuntu Wiki (community-edited website)|||1602647046960000|1602647046960000|6A8e4nK1328m|0|1

This isn't really usable, by any stretch, at the moment.  But a little googling, and I found a site called purposeful.co.uk, whom posted a nice shell script which would do the job for you.  
The link to this is: http://www.purposeful.co.uk/software/places2bookmarks/

The script is on there for you to copy to your system, I am not going to repost it here.  But, for readability, I will say that I modified a single line in the script.  Here is the modified line:

echo "$3  <A HREF=\"$url\">$title</A><br />"

As you can see, I added a '<br />' so each link would be on its own line, making things much nicer, and one entry per line.


Once you have the script in place (and the modification, if you so choose), you can run it like this: 

 ./places2bookmarks.sh ./places.sqlite > bookmarks.html

You will need to ensure that you replace the './' with the correct path to the files, if that is incorrect.  For me, I had them in the same directory

This doesn't seem huge thing, and sure, you could use the gui, but that is not me.  I prefer to do things on the command line and figure them out. Besides, I had fun doing it and learning about it, and thought I would share with everyone in case anyone else found it fun and fascinating.

Enjoy!

Thursday, February 28, 2019

Untracking files in git

In one of my current projects, pyblueprint-py3, I have a config file.  This file gets edited locally after cloning the repository, and those changes get seen by git.  That is a problem.  This is a file that is distributed with default values that the user edits, and git does not need to know about these changes.  But, how do we tell git not to care about changes to a file?  Good question, and here is the answer.

For your config file (or whatever file it is that you want to prevent git from tracking), create it in your repository, and then issue the following command:

    git update-index --skip-worktree file_name

The file_name should include the path if it isn't in the current directory (ie: its in a subdirectory).

If the file that you don't want to track is already on git's radar, then you can do the following:

    git rm 
    git rm -r --cached 
    git commit -m'- Cleaning up and removing file to be ignored'
    git push

Then, add the file and edit the conf file with the generic placeholder and do this:

    git add 
    git update-index --skip-worktree 
    git commit -m'- Committing new, untracked file'
    git push

One thing to remember, is that when someone clones your repository, before they do anything, they will need to reissue the 'git update-index' command above for the specific file you untracked.  That should be noted in your documentation.



pyblueprint-py3 Now Exists

What a crazy 3/4 of a year its been.  The last company I worked for kept going down hill for the employees with a ton of management change-over, which was in no way good.  Eventually, things had to change and I needed to get back to a happy place.  At the turn of the year I started a new job, and BOY, am I happier.  While I am still ramping up at my new place of employment, I am certainly having a lot more fun here than I did the last couple years at my old job.  Considering what is going on over there, I do not envy my old colleagues.

Enough about the craziness.  I recently sat down and took the time to convert the pyblueprint project over to using Python3.  Not only that, there is a new option for specifying a language.  It defaults to python, but it will create a directory structure for whatever language you choose.  But, python and Ruby have support for the virtualenviroment that this project sets up (as does Go and NodeJS, but I haven't yet put together docs and such for that.  That is coming though.  I migrated the project to a new name, call pyblueprint-py3.  Feel free to download it and play with it.  Hopefully it works for you.  If there are issues, please open a ticket in gitlab under the project.  Also, I am open to feature requests, should anyone have any. 

Hopefully I can get more additions on to this project in the near future. 



Friday, July 06, 2018

A new home for my projects and a project update

New Home

I know that there is always an expectation that, if you are a developer of some sort, you store your code into Github (at least that is the first one that comes to people's mind).  I have a Github acct, but, because their private repos are not free, I haven't truly used them for storing my repos.  And considering that Microsoft now owns them, I really won't be using them for my code.

Up until now, I have relied on Bitbucket (currently owned by Atlassian), as it offers unlimited free private and public repos.  We have both Bitbucket and Gitlab at my work (local installs for both).  After all of the increased fanfare for Gitlab (after the Microsoft purchase of Github) and my exposuer to it at work, I have made the switch to store my repos there as well.  Unless they change their policies, I will probably be with them quite a while.  


Project Update:

I did want to say that I have migrated my 'pyblueprint' project over to Gitlab.  Thankfully Gitlab makes the migration to them painless, and it was just a click of a couple of buttons to get the repo moved over.  The 'hard part', if you will (although not really), was fixing my git remote to point to the new repository location.  

I do want to say that I have added support to the project for creating a repository up in Gitlab.  Seeing as how I am now using it, why not.  :)  While testing out the application to ensure everything was still working afterwards, I discovered that my checks were not working.  Why?  Because the maintainers of 'pip' decided that, as of pip 10.0.0, they no longer support pip bing used programatically.  If you drop into IDLE, do an 'import pip' and then do a 'dir(pip)', you will now notice that there are no methods associated (aside from the underscore methods).  This completely broke my checks as I had relied on that useful bit.  No worries though, the checks are fixed and functioning as expected.  

Also, if anyone is using this application (don't know if anyone is) and they would like support for an alternative code repository, please feel free to open an enhancement ticket and I will see about adding support for it.  

That's it for now, happy coding!
 
Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.