Politics

xrayspx's picture

Playlists

Music: 

Dr. Dre - Nuthin' But a G' Thang

I had a request to share some playlist management stuff so I thought I should explain myself. I've got a significant CD collection, and a somewhat-significant collection of TV shows. This is fine on its own, but lots of media is pretty worthless without well curated playlists that you really don't have to think about. So I built Spotify, MTV and Syndicated TV.

* NOTE: If you have a better way to do any of this let me know and I'll fix it. I particularly have the sense, which is not backed up by my testing, that "sort -R" isn't great.

Music's easier so we'll start there. I use Strawberry to manage my music. This was all running under Clementine and aside from some DB schema changes, the scripts are portable between them.

Until relatively recently I was never a big fan of "star" or "heart" ratings, but Clementine/Strawberry will store this metadata in the MP3 itself so I should be able to quickly recover if I lose my music database. In the app I have a few Smart Playlists like 3-Stars, 3 Stars + (This is 3, 4 and 5 star tracks), 4-Stars, 4-Star + and 5 Stars. To use 4 Star as an example, the rules look like this:

Match every search term (AND)
Rating - Greater than - 3.5 Stars
Rathing - Less than - 5 Stars
Ratin - Not Equals - 5 Stars
Length - Greater Than - 8 Seconds

That results in a playlist of 8423 songs with ratings between 4 and 4.99 stars. There was a bug in Clementine which I got fixed where ratings could exceed 5, so I'm a little careful to deal with weirdo cases, but it's pretty simple. I also have a bunch of manually selected playlists, so like an '80s one, '90s, and "Barn Radio". Barn Radio is our catch-all for the ubiquitous music we heard from the late '70s through late '80s. For Natalie that was largely with her dad in the dairy barn, for me it was the music of my 2 hours on the bus every day.

Anyway, I have all these .m3us stored in a folder along with my MP3s called "playlists_base". These are used by a nightly playlist generator that pulls ~200 tracks and makes daily playlists running 8 or 10 hours each. The reason for this is that streaming software such as Airsonic-Advanced kind of chokes on massive playlists. It could be Airsonic itself, it could be populating the mobile client, I don't really know or care, other than to say it works great with list sizes under about 1000 tracks or so, so I keep them shorter.

The x-Star playlists are all built from the database like this 4 Star + playlist below. You can see it do a couple of different Star Rating DB queries, dump out the tracks to $playlist_tmp.m3u, then cat that file and do a random sort to generate the final version. It's pretty easy to adjust the mix based on ratings, so if I wanted to weight high-rated tracks I could do that by adjusting how many tracks of the 200 are returned by each search:


#!/bin/bash

rm /Volumes/Filestore/CDs/playlists/4\ Stars\ +.m3u

i=1

while [ $i -le 100 ]
do

### Switching from Clementine to Strawberry ###
#       file=$(sqlite3 /var/tmp/clementine.db "select filename from songs where rating > "0.9" order by random() limit 1;" | awk -F "file://" '{print $2}')
        file=$(sqlite3 /var/tmp/strawberry.db "select url from songs where rating > "0.9" order by random() limit 1;" | awk -F "file://" '{print $2}')

        ### Clementine data encodes special characters and accent marks and stuff so I'm using
        ### Joel Parker Henderson's urldecode.sh to undo that: https://gist.github.com/cdown/1163649

        data=$(/home/xrayspx/bin/urldecode.sh "$file")
        if [ -f "$data" ]
        then
                ### Have to escape leading brackets because grep treated it as a range and would allow duplicates ###
                ### Can't do that in "data" because \[ isn't in the filename so they'll fail ###

                escaped=$(echo "$data" | sed 's/\[/\\[/g')
                #echo "$escaped"

                ### Avoid duplicates
                match=$(grep -i "$escaped" /var/tmp/4-star-tmp.m3u)
                if [ -z "$match" ]
                then
                        echo "$data" >> /var/tmp/4-star-tmp.m3u
                        ((i++))
                fi
        fi
done

i=1

while [ $i -le 100 ]
do
### Switching from Clementine to Strawberry ###
#        file=$(sqlite3 /var/tmp/clementine.db "select filename from songs where rating >= "0.8" and rating          file=$(sqlite3 /var/tmp/strawberry.db "select url from songs where rating >= "0.8" and rating 

        ### Clementine data encodes special characters and accent marks and stuff so I'm using
        ### Joel Parker Henderson's urldecode.sh to undo that: https://gist.github.com/cdown/1163649

        data=$(/home/xrayspx/bin/urldecode.sh "$file")
        if [ -f "$data" ]
        then
                ### Have to escape leading brackets because grep treated it as a range and would allow duplicates ###
                ### Can't do that in "data" because \[ isn't in the filename so they'll fail ###

                escaped=$(echo "$data" | sed 's/\[/\\[/g')
                #echo "$escaped"

                ### Avoid duplicates
                match=$(grep -i "$escaped" /var/tmp/4-star-tmp.m3u)
                if [ -z "$match" ]
                then
                        echo "$data" >> /var/tmp/4-star-tmp.m3u
                        ((i++))
                fi
        fi
done

cat /var/tmp/4-star-tmp.m3u | sort -R > /Volumes/Filestore/CDs/playlists/4\ Stars\ +.m3u

rm /var/tmp/4-star-tmp.m3u

Those Star Rating lists are called at the beginning of my overall static playlist script, but the Barn playlist and other manually selected ones are built from the "playlists_base" directory. I basically just edit those .m3us in place with Strawberry as we add CDs. They just the files, do a random sort and pull the top 200. This will use any .m3u in .../playlists_base/ and make a daily file from it:


#!/bin/bash

#scp xrayspx@pro:~/.config/Clementine/clementine.db /var/tmp/

### Switching between Clementine and Strawberry ###
#cp /Volumes/Filestore/CDs/playlists_base/clementine.db /var/tmp/

cp /Volumes/Filestore/CDs/playlists_base/strawberry.db /var/tmp/

/home/xrayspx/bin/3-star-playlist.sh
/home/xrayspx/bin/4-star-playlist.sh
/home/xrayspx/bin/5-star-playlist.sh
/home/xrayspx/bin/get-the-led-out.sh

ls /Volumes/Filestore/CDs/playlists_base/*.m3u > /Volumes/Filestore/CDs/playlists_base/m3us.txt

while IFS= read -r file
do

        filename=$(echo $file | awk -F "/Volumes/Filestore/CDs/playlists_base/" '{print $2}')

        echo Filename: $file

        rm "$file.full"
        rm "$file.scratch"
        rm "/Volumes/Filestore/CDs/playlists/$filename"

        ###Testing a change since Strawberry creates playlists without EXTINF lines ###
#        array=`grep EXTINF "$file" | sort | uniq`
        array=`grep -v EXTINF "$file" | sort | uniq`

        printf '%s\n' "${array[@]}" | sort -R > "$file.full"
        head -n 200 "$file.full" > "/Volumes/Filestore/CDs/playlists_base/$filename.scratch"

        n=0
        while IFS= read -r extinfo
        do
#       echo $extinfo
                term=`echo $extinfo` # | cut -d "," -f 2-`
#       echo $term

 ###Testing a change since Strawberry creates playlists without EXTINF lines ###
 # grep -A 1 -m 1 "$term" "$file" >> "/Volumes/Filestore/CDs/playlists/$filename"

        grep -m 1 "$term" "$file" >> "/Volumes/Filestore/CDs/playlists/$filename"
        done 

        rm "$file.full"
        rm "$file.scratch"

done 

rm /var/tmp/clementine.db
rm /var/tmp/strawberry.db

For TV shows it's a bit more complicated. I've got individual scripts for things like Sitcoms, Saturday Morning Cartoons, Buddy-Cop shows, Nick-at-Nite, etc. Each script uses a text file which just lists the relative path to the directories I want to randomize. I just read in that text file then scan each directory and build an array that again I sort -R and dump in an m3u. You'll see a couple of my conventions here, like the "dvd_extras" folders I use for any extras that I want to keep but don't want to have show up in the mix, as well as a bunch of other crap I grep out.

This script references "./.sitcoms.txt", which looks like this:


./Archer (2009)
./30 Rock
./Absolutely Fabulous
./Alexei Sayle's Stuff


#! /bin/bash

array=$(
while read line
do
        find "$line" -type f;
done < .sitcoms.txt
)

printf '%s\n' "${array[@]}" | sort -R | grep -v -w "batch" | grep -v dvd_extras | grep -v "./$" | grep -v "\.m3u" | grep -v -i ds_store |
 grep -v "\.nzb" | grep -v "\.nfo" | grep -v "\.sub" | grep -v "\.sfv" | grep -v "\.srt" | grep -v -i "\.ifo" | grep -v -i "\.idx" |
 sed 's/^/..\//' > ./1\ -\ Playlists/Sitcoms.m3u

This dumps out to a folder called "1 - Playlists" inside my TV Shows directory, just so it shows up first. There's a folder in there for Blocks as well, in which I create blocks of 10 random episodes of a bunch of shows. This is built to replicate like TBS/TNT/USA in the evening where you just sit and watch a block of whatever is on. In practice I do this wrong and tend to be too picky about these and just watch blocks until I've worked my way through a whole series and wind up tired of it forever.

One thing I do for things like Nick at Nite and overall Sitcom lists and stuff is that I mix in commercials. I don't do this very well though, I just treat my directory of commercials like any other TV show. I'd rather do "pull a TV show, toss in two commercials, repeat", but I'm not there yet I guess.

The last type of lists I build are for music videos. I break this into a few different playlists, one overall catchall that pulls in all videos, a playlist for MTV 120 Minutes, and one for "Arcade / Pizzeria" music. Basically the ubiquitous music you'd hear in a pizza shop or arcade in the '80s or '90s. I do the same commercial thing here as well.

Example:


#! /bin/bash

array=`find ../120\ Minutes -type f;
find ../../../Commercials -type f`

printf '%s\n' "${array[@]}" | sort -R | grep -v dvd_extras | grep -v "./$" | grep -v "ERRORS$" | grep -v "\.sh" | grep -v "\.m3u" |
 grep -v -i ds_store | grep -v ".nzb" | grep -v ".srt" > 120\ Minutes.m3u

xrayspx's picture

Photo Backup

Music: 

I'm just sticking this here because it seems Mr. Santorum is reportedly expending some effort to get this photo removed from anywhere it's found on the Internet.


So here's a photo of Rick Santorum with his arm around Russian spy and notorious honey trap Maria Butina. I mean, I'm not saying he fucked her, though it seems many other Republicans did. So if he didn't fuck her, then Santorum either missed out or dodged a bullet depending on how you look at it.




xrayspx's picture

My Life Is Going To Suck Without Net Neutrality

Music: 

There are so many things I do which are likely to suffer with Net Neutrality's loss.

I run my own mail, web and cloud sharing services on a VPS that I maintain. Owncloud syncs all my devices, I use IMAP and webmail. I also run lots of "consumer" stuff for myself. I own 2500 CDs which I've ripped and share for my own personal use. I have playlists. I can connect with DAAP from my phone, and listen to my own CD collection, music I have paid for, Spotify style. I know people are saying "Spotify will work just fine", but what if I don't want to use Spotify?

This is all encrypted, personal connections. Nothing illegal is happening here. I'm not filesharing or streaming Torrents or any other grey-area services. It's just all my personal stuff, owned and manually copied myself, sharing to myself. No one gets ripped off here.

I can plug my Amazon Fire stick or Raspberry Pi into any TV and use Kodi to stream my own MP3s or movies, etc. I can use it to watch Amazon Prime or Netflix as well. Kodi also has a wealth of plugins to watch content from sources such as the PBS website. We all can watch Nova, or Julia Child, or even Antiques Roadshow over the Internet, for free, legally. This may all suffer when backbone providers and local ISPs can both decide which packets have priority over other traffic. PBS could be QOS'd out of the budgets of millions.

(Note *)I don't own a Nest or any other IOT garbage, but I have toyed with the idea of building my own, running on infrastructure I build. I don't want Google to know what temperature my house is right now. And I don't want some mass hack of 500 Million Nest users or idiot IOT Lightbulbs to let some Romanian turn my furnace off in the middle of February either.

So yeah, losing Net Neutrality could effectively disable all of this. Small hosts like me could be QoS'd off of the Internet entirely, unless we pay extra /at both ends/. Pay my hosting provider to pay their backbone providers to QoS my address at a decent speed. Then pay my consumer ISP to QoS my traffic so I can reach "The Good Internet", like they have do in Portugal.

This is going to cut my lifeline to my own data, hosted by me on my own machines. Am I going to have to pay an additional "Get Decent Internet Access Beyond Google, Spotify, Facebook and Twitter" fee to the Hampton Inn just so we don't get QoS'd away from our own stuff? It's bad enough that the individual hotel can effectively do this already today, but the hotels are at least limited by the fact that they're in competition with each other and if they have ridiculously shitty Internet that you can't check your mail over, well people would notice that. Backbone providers pretty much have no such direct consumer accountability. No one's going to say "well, fuck that I'm not going to route over AT&T anymore", they might say "Hilton has shitty Internet, I'm going to Marriott".

Some of the most demoralizing part of this is that the rule-makers just don't get it. I already know they don't care, but former FCC Chair Michael Powell's statement, which boils down to "You can still use Facebook, (Amazon) Alexa, Google and Instagram, just like you can now" is missing the point either deliberately or purposefully. That most "consumers" will be fine isn't the point. The point is that everyone be equal, and all traffic be routed equally.

* The risk to my information is proportional to the value an attacker places on the information. Could a state actor target my email server and read my mail? Yeah, the Equation Group or Fancy Bear or some Eastern European ID theft ring could probably exploit some flaw in whatever software serves my VPS, or flat out order the ISP to give them access to my stuff, but why? What does the NSA gain by ransacking my mail server? Not much. How about criminal attackers? However they /would/ expose 1.5 Billion Yahoo accounts all at once, and have that entire corpus of mail to search against, plus passwords they could use to try and attack everyone's bank account all at once.

xrayspx's picture

You know what, no, they don't

Music: 

Because if people could remember what 100 years ago Earth was like, they'd know that the best things to happen in the last 100 years are based around the idea that if we all work together, then when we're old, we will take care of each other. And when we're young, rather than work like adults, we will teach our children with the collective knowledge of our species so we can continue to advance. We can afford to take care of those who can't work like the rest. Too much of the time, we choose not to take care of those people.

xrayspx's picture

Do people really not understand how good things are?

Music: 

One thing which has bothered me as much as anything about the right wing populist uprising this year is that if people could even remember to 100 short years ago, literally zero of what they see as their daily life to which they're entitled even existed.

xrayspx's picture

James Bond Accidentally Killed Democracy

Music: 

It's the best I can do folks. This is the most obvious, if accidental conspiracy I can think of.

Fixed Tags:
xrayspx's picture

2016 Third Party Voters

Music: 

In 2016, a vote for a third party is a vote to legitimize Trump's fascist vision.

As I sit here watching a Donald Trump rally, and my fellow Granite Staters chanting "Build That Wall" while a stream of consciousness volley of nonsense, lies and unconstitutional threats comes out of this horrible man, I'm reminded of exactly how close the race is here in New Hampshire.

Far from being utterly repudiated, we're looking at fairly even poll numbers in New Hampshire. Think about that.

Fixed Tags:
xrayspx's picture

KRS-One in Manchester

Music: 

Clearly I need to post more music stuff, last night we went to KRS-One.

... and it seems that my camera has fucked every photo I've taken in the last month. So enjoy oddly-cropped KRS-One:

"Mic check, mic check, louder, louder more! more! ('Turn them shits up!') whoops, shit, no power":

"But no problem, we don't need power":

Temple of Hip-Hop:

xrayspx's picture

Simple location aware ssh tunneling for Chrome (Mac)

Music: 

Hall and Oates - Private Eyes (Seriously, it just came on randomly)
and really, just as I finished formatting the stupid script, Big Brother from Humanwine was playing.

This is both a nice toy to have in a Big Brother Is Watching sense, and a glaring example of why one should never log in and use a Mac (or any other system obviously) as an Administrator. Just have a Regular Guy account, and escalate to Administrator/Root when needed. For example, this tool could be inserted by a script to cause all your browsing traffic to route through a proxy server of an attackers choosing. If you're not running as an Administrator, you can't write the file without escalating. (Example of the risk, though it wouldn't help here, since there is LCE to root...goddammit Apple...)

I had a use case recently where I wanted to have multiple copies of Chromium start in different profiles and with different proxy settings. I'm getting to the point at which I don't think that's really feasible, in that any new instance will assume the proxy settings of any already running instance.

BUT, I did get some cool location aware-ish proxying set up. Since one use case involves laptops, I'd like to see it use a local proxy when I'm home, and a remote proxy when I'm not at home (hosted VPS for instance).

I'm using ssh to set up a SOCKS5 proxy, and push all traffic including DNS through the tunnel, ssh'ing to different hosts based on different local system IPs. I have it checking en0 and en1 and if their IPs match my home subnet, it ssh's to a local system, if they are anything else, it will run against a publicly hosted system to which I can ssh.

Next step is to clean up after itself, so when you run Chromium (or Chrome), it will detect IPs, ssh to the appropriate host, and connect using that tunnel. When Chromium closes, it cleans up the SSH session so it's not just hanging around.

To use - Have a local and remote host you can ssh to using keys, and which allow you to forward. On the Mac, navigate to /Applications/Chromium.app/Contents/MacOS/. Rename Chromium to Chromium-bin. Drop this script in, chmod appropriately, and name it Chromium. Now when the Chromium app is run, it runs our script to set up proxies and launch the browser:


#! /bin/bash

ip0=`ifconfig en0 | grep -v inet6 | grep inet | awk '{print $2}' | awk -F "." '{print $1"."$2"."$3}'`
ip1=`ifconfig en1 | grep -v inet6 | grep inet | awk '{print $2}' | awk -F "." '{print $1"."$2"."$3}'`

if [ -z "$ip0"  ]
  then
   if [ "$ip1" = "192.168.30" ]
     then
       ssh -C2qTnN -D 8181 username@192.168.30.241 &

       proxypid=`jobs -p`
       /Applications/Chromium.app/Contents/MacOS/Chromium-bin --proxy-server="socks5://127.0.0.1:8181" --host-resolver-rules="MAP * ~NOTFOUND, EXCLUDE 127.0.0.1" --profile-directory=Tunnl 2>&1 /dev/null

       kill $proxypid

      else

        ssh -C2qTnN -D 8181 username@publichost.com &

        proxypid=`jobs -p`
        /Applications/Chromium.app/Contents/MacOS/Chromium-bin --proxy-server="socks5://127.0.0.1:8181" --host-resolver-rules="MAP * ~NOTFOUND, EXCLUDE 127.0.0.1" --profile-directory=Tunnl 2>&1 /dev/null

        kill $proxypid

      fi

  elif [ "$ip0" = "192.168.30" ]
    then
      ssh -C2qTnN -D 8181 username@192.168.30.241 &

      proxypid=`jobs -p`
      /Applications/Chromium.app/Contents/MacOS/Chromium-bin --proxy-server="socks5://127.0.0.1:8181" --host-resolver-rules="MAP * ~NOTFOUND, EXCLUDE 127.0.0.1" --profile-directory=Tunnl 2>&1 /dev/null

      kill $proxypid

  else

      ssh -C2qTnN -D 8181 username@publichost.com &

      proxypid=`jobs -p`
      /Applications/Chromium.app/Contents/MacOS/Chromium-bin --proxy-server="socks5://127.0.0.1:8181" --host-resolver-rules="MAP * ~NOTFOUND, EXCLUDE 127.0.0.1" --profile-directory=Tunnl 2>&1 /dev/null

      kill $proxypid

fi

xrayspx's picture

Hmm. So that's how it is in their family

Music: 

Shriekback - Malaria

TL;DR: Here is how to restore DJ to iTunes, as much as possible

A few months ago, Apple maliciously broke iTunes in several really specific ways, one of which was to drop the DJ functionality, which is basically how I would listen to music.

Reading a thread on JWZ's site this issue, among others, I posted my somewhat-fix for the issue. And it is. A "somewhat" fix. It acts pretty much like DJ used to act, but for two problems. You can't drag things from a window with your whole collection into your "DJ" window (Cause hey, ONLY ONE WINDOW NOW), and besides, I had to create a Smart Playlist to fix it, and you can't add to a smart playlist anyway. There is "Play Next", which I guess works.

My other main gripe with this is that when I hit Next to skip a track, usually it removes it from the top of the playlist, but often enough to annoy the fuck out of me, it doesn't, and I have to go back in and clean up the top of my list a few times a day. Worse, songs I've skipped will come back up in the mix sooner than I would otherwise want them to, since iTunes doesn't know I've skipped them.

I remember reading somewhere that there was a discussion once about how to make iTunes mark something as "Skipped", or at least what the secret parameters are that cause things not to become "Skipped". So tonight it annoyed me enough to hunt around, and of course, the very first hit was back to a different JWZ post from exactly three years ago this week, complaining about this exact skipping thing.

Of course he didn't get a satisfactory answer, because he almost never gets a satisfactory answer to exactly what he asked. It looks like if you skip between 2 and 20 seconds into the song, and don't hit pause ever, it will show as Skipped. Neat.

His Herp Derp checkbox was the only thing that made any of this sane for me in this case.

To mostly restore iTunes DJ, do the following:

Click + at the bottom left of the iTunes window and create a new Smart Playlist. I named mine "DJ-ish".

Match All of the following rules:

  • Last Played not in the last 1 days -- Or however long you want to go between repeats
  • Last Skipped not in the last 2 days -- This will make iTunes clean up most songs you skip using the Next button.
  • Limit to 100 items selected by Random -- or however many upcoming tracks you want it to pull at a time
  • Match only checked items -- Unless you want iTunes to randomly play songs you've explicitly told it you don't want to hear by un-checking them
  • Live Updating

It's pretty simple to get most of that functionality back, but you know what would have been simpler? NOT REMOVING IT.

Pages

Subscribe to RSS - Politics