Sunday 10 September 2017

I lobster and never flounder.

Had trouble with some of the piscine puns in the song so here come my annotations


I was a cook and she was a waitress
Down at the Salty Sam Seafood Café
And somewhere 'tween the clam juice and the seaweed salad
Some little shrimp[someone insignificant and possibly short in stature] just lured her away

Oh, I lobster[lost her] and never flounder[found her]
He wrapped his line around her
and they drove off in his carp[car]
Oh, I lobster[lost her] and never flounder[found her]
I octopus[oughta push] his face in
Eel[he'll] only break her heart
(Note: push someone's face in = to hit on the nose, the idiom is mentioned in the Oxford English Dictionary in the entry for the word "push")

I said, "Just squid[quit] and leave me
for that piano tuna[tuner]
if you want to trout[try out] someone new"
She was the bass[best] I ever had
Now my life has no porpoise[purpose]
Oh, my cod[god], I love her, yes, I do.

(refrain)

... now trout and sea if ayu are koi enough to smelt out the rest

Sunday 9 July 2017

Ubuntu Dell sound disappearing

So this should fix the sound disappearing problem that some people might be experiencing. It happens usually when you insert/remove headphones but sometimes seemingly at random.

1) Curse at Dell and Linux
2) sudo alsactl kill quit
3) Curse at Dell and Linux some more
4) alsactl restore 0
5) Intimidate your laptop by saying what a nice cutting board it would make once your new Mac or Surface arrives.
6) Work for a few hours in peace, then go to step 1

Now you have sound.

Friday 5 May 2017

leaks and MallocStackLogging is pretty awesome

Keep in mind it may trigger false positives if the allocated memory address is for some WEIRD reason computed indirectly.

🙈 --with-malloc-mempools

Sunday 21 February 2016

Monday 21 December 2015

Running DTruss on OS X El Capitan



Dtruss in OS X, much alike strace in Linux, is a very useful tool providing the capability to look at what a program is doing (more precisely, how it interacts with the operating and file system) without having to debug it on the source code level. In a recent system update called El Capitan, however, the tool stopped operating with default settings due to the newly implemented System Integrity Protection mechanism. Trying to run dtruss now results in an error:
$ dtruss ls
dtrace: failed to initialize dtrace: DTrace requires additional privileges
$ sudo dtruss ls
dtrace: failed to execute ls: dtrace cannot control executables signed with restricted entitlement

Some solutions on the Internet suggest to fully or partially disable System Integrity Protection to allow dtruss to do its bidding. However, such dire measures are not really necessary in most cases, since all that is really needed is to let it know that the program you're running is signed with no "restricted entitlement", i.e. which is to be trusted and allowed to be "spied" on.

This short step-by-step guide shows how to run dtruss without meddling with SIP (and, thus, potentially compromising the computer's security).
We assume that we want to run dtruss on the standard command line utility ls. To run dtruss on any other application, just replace the corresponding lines with the path to your application.

1) Create a copy of the standard sudo utility in a non-protected directory and re-sign it with an ad-hoc signature. This is needed so that we could run our application with standard user privileges (dtruss itself requires administrator privileges).
$ sudo mkdir -p /usr/local/bin # the directory may already exist on your system
$ sudo cp /usr/bin/sudo /usr/local/bin/sudo
$ sudo codesign -fs- /usr/local/bin/sudo

2) Now, re-sign the application you want to trace with an ad-hoc signature. Note: it will not work if the application is located in one of the standard binary directories which are protected (/System, /bin, /sbin, /usr/bin). In that case you need to copy the application somewhere else.

$ cp /bin/ls /usr/local/bin # this is obligatory only if the binary is located in a protected directory, or if you want to preserve the signature on its original copy
$ sudo codesign -fs- /usr/local/bin/ls

3) Finally, you're good to run struss!
$ sudo dtruss -f /usr/local/bin/sudo -u `whoami` /usr/local/bin/ls
...
43024/0x14e0f8:  fchdir(0x4, 0x7FE2DC002600, 0x1000)         = 0 0
43024/0x14e0f8:  close_nocancel(0x4)         = 0 0
43024/0x14e0f8:  fstat64(0x1, 0x7FFF5DB92D88, 0x1000)         = 0 0
43024/0x14e0f8:  ioctl(0x1, 0x4004667A, 0x7FFF5DB92DCC)         = 0 0
43024/0x14e0f8:  write_nocancel(0x1, "AUTHORS\t\tChangeLog-2014\tREADME-alpha\tbootstrap.conf\tdoc\t\tm4\t\tthanks-gen\n@\004\b\0", 0x48)         = 72 0
43024/0x14e0f8:  write_nocancel(0x1, "BUGS\t\tMakefile.am\tTHANKS.in\tbuild-aux\tgnulib\t\tpo\n\004\b\0", 0x31)         = 49 0
43024/0x14e0f8:  write_nocancel(0x1, "COPYING\t\tNEWS\t\tbasicdefs.h\tcfg.mk\t\tgnulib-tests\tsed\n@\004\b\0", 0x34)         = 52 0
43024/0x14e0f8:  write_nocancel(0x1, "COPYING.DOC\tREADME\t\tbootstrap\tconfigure.ac\tlib\t\ttestsuite\n\b\0", 0x3A)         = 58 0
...

Used sources:
http://stackoverflow.com/a/33776939/161934
https://sourceware.org/gdb/wiki/BuildingOnDarwin

P.S. With this approach the first few system calls will belong to sudo and not your app. Just skip them. Or try suspending the process at its start and attaching to it with dtruss (I'm still figuring out the best way to do that myself without meddling with the dynamic linker...).
P.P.S. As it may be seen, this post was written by someone not very familiar with OS X. I'll greatly appreciate any comments on potential mistakes and misconceptions encountered in it.
P.P.P.S In this article we signed the application with an ad-hoc signature. If you want to create your own signature and sign the application with it, do the following:
  • Run the Keychain Access application (⌘+Space => "Keychain Access" => Enter)
  • Pick "Keychain Access" => "Certificate assistance" => "Create a Certificate"
  • Set the type of the certificate to "Code" and give it an arbitrary name "your_certificate_name" which you will need to re-enter later. Your username should do.
  • Create the certificate. 
  • Then, re-run codesign as indicated above, replacing "-" with your own signature.

Monday 4 May 2015

How much memory will it take to store a single character string in Python 3?

Been investigating an unexpected bloat in one of my Python programs and stumbled upon this fairly surprising result. As I was storing a lot of short strings, I decided to see how much space a one character long string ('x') takes. As it turns out, a whole friggin' lot! Below I'm providing the structure used to store a string object in CPython, with all the macros expanded. Every string object gets a separate copy of this structure. See unicodeobject.h and object.h for details.

The data below pertains to the 64-bit interpreter version, on the 32-bit version the total size should be smaller.

typedef struct {
    /* _PyObject_HEAD_EXTRA */
    /* me: All objects are linked in a list. */
    struct _object *_ob_next;
    struct _object *_ob_prev;
    /* PyObject_HEAD */
    /* me: Py_ssize_t is 4 bytes long. */
    Py_ssize_t ob_refcnt
    struct _typeobject *ob_type;
    Py_ssize_t length;          /* Number of code points in the string */
    /* me: Py_hash_t is 4 bytes long. */
    Py_hash_t hash;             /* Hash value; -1 if not set */
    struct {
        unsigned int interned:2;
        unsigned int kind:3;
        unsigned int compact:1;
        unsigned int ascii:1;
        unsigned int ready:1;
        unsigned int :24;
    } state;
    wchar_t *wstr;              /* wchar_t representation (null-terminated) */
} PyASCIIObject;

Total = sizeof(PyAscIIObject) + strlen('x') = 48 + 2 = 50. The same number can be obtained from within Python by using the standard sys.getsizeof() function:
>>> sys.getsizeof('x')
50

50 bytes to store a single character string. Yikes!

P.S. One seemingly obvious optimization would be to lay objects in memory in a way that logically adjacent objects are close to each other in memory, so instead of using 8-byte pointers 2/4-byte offsets would suffice. (I'm a newbie though, so I have absolutely no clue how feasible that would be in CPython.)

Friday 6 February 2015

Making custom versions of packages in Ubuntu/Debian

Had to make some changes in the source code of a couple of packages on my system and decided to document the process for future reference. Surprisingly, it's way easier than I thought it would be.

As an example, suppose we want to make the default graphical viewer Eye of Gnome not jump back to the first picture after the last one in a directory (and, similarly, not jump in the opposite direction, either).

First, we need to get the source code for the program
$ mkdir eog; cd eog; apt-get source eog

Then let's install all the needed dependencies to compile the package:
$ sudo apt-get install build-dep eog

All right, time to make the changes we need. In this case, we need to slightly modify the corresponding part in eog_thumb_view_select_single() in src/eog-thumb-view.c to look like this:

        case EOG_THUMB_VIEW_SELECT_LEFT:
            gtk_tree_path_prev (path);
            break;
        case EOG_THUMB_VIEW_SELECT_RIGHT:
            if (gtk_tree_path_get_indices (path) [0] != n_items - 1)
                gtk_tree_path_next (path);
            break;

Now that we've made the changes, let's make our custom package:
$ dpkg-buildpackage -b

The program has been compiled and assembled in one neat package. All that is left is to install it, replacing the standard version:
$ sudo debi

Hooray, we have our custom version of Eye of Gnome installed and operational!

Now if I only knew a way to apply patches and rebuild the packages automatically every time they get an update..

Sunday 16 November 2014

Source-compiled Wine with ALSA output plays static in Pulseaudio under Ubuntu.

PULSE_LATENCY_MSEC=30 wine
# =1 works in some cases, doesn't on others
http://arunraghavan.net/2013/08/pulseaudio-4-0-and-skype/

Friday 5 September 2014

Index project with ctags and cscope with a single button press in emacs.


I should probably comment on the madness happening below, but I'm too tired at the moment, and, more importantly, I don't think anyone needs it anyway.

The short and messy description is that you press f9 and both ctags and cscope databases are built for your project. Pretty convenient.

To make it work with the project of your choice, you need to describe your project in the known-projects list (it's already filled with some examples).

The first part is a list of directories that designate the root of the project (so that you can run the command from any internal dir).

The second part is a blacklist that contains names and paths that should not be indexed, e.g. you would want to blacklist the huge test collection in GCC which will inflate your database tremendously and add an incredible amount of duplicated names, or the equally humongous Linux driver collection (unless you're a driver developer, duh).

1-2 auxiliary subroutines may have been copied from various places on the Internet. The code may hang your editor if you have cyclic links. Also, I can't program in elisp, although that must be strikingly obvious without me saying that.

;; Define the core dir trappings and blacklisted directories here
(setq known-projects '((("gcc" "config" "libgcc" "libiberty") ("/testsuite/" "/test/" "/build/"))
               (("mono" "mcs" "eglib" "docs" "tools") ())
               (("DROD" "DRODLib" "DRODUtil") ())
               (("arch" "drivers" "include" "kernel") ("/drivers/")) ; Linux
 

               (("README" "INSTALL") ("/.pc/")) ; Generic software project #1 (e.g. emacs)
               (("README") ("/.pc/")) ; Generic software project #2.
               ((".") ())          ; Oh well, let's at least index the current dir
               ))              ; if everything else fails.

(defun index-project () (interactive)
  (defun dump-vars-to-file (varlist filename)
    "simplistic dumping of variables in VARLIST to a file FILENAME"
    (save-excursion
      (let ((buf (find-file-noselect filename)))
    (set-buffer buf)
    (erase-buffer)
    (loop for var in varlist do
          (princ (concat var "\n") buf))
    (save-buffer)
    (kill-buffer))))
 
  (defun lookup-project (projects)
    (let ((lookup-failed t) (blacklist nil) (remaining-trappings nil) (lookup-path nil) (project nil))
      (while (and (consp projects) lookup-failed)
    (setq lookup-path buffer-file-name)
    (setq project (pop projects))
    ; (message "Trying project %s" project)
    (while (and (> (string-width lookup-path) 1) lookup-failed)
      ;; For some reason (file-exists-p "main.c/.") = t!
      (when lookup-failed
        (setq lookup-path (replace-regexp-in-string "/[^/]*$" "" lookup-path)))
      ; (message "Trying path %s" lookup-path)
      (setq lookup-failed nil)
      (setq remaining-trappings (nth 0 project))
      (setq blacklist (nth 1 project))
      (while (and (consp remaining-trappings) (not lookup-failed))
        (if (not (file-exists-p (concat lookup-path "/" (pop remaining-trappings))))
        (setq lookup-failed t)))))
      (when lookup-failed (setq lookup-path nil) (setq blacklist nil))
      (list lookup-path blacklist)))

  (let ((project-info (lookup-project known-projects))
    (project-root nil) (project-blacklist nil))

    (setq project-root (nth 0 project-info))
    (setq project-blacklist (nth 1 project-info))

    (message "Project found at %s. Scanning files... (excliding %s)" project-root project-blacklist)

    (when project-root
      (defun is-path-blacklisted (path blacklist)
    (while (and blacklist (not (string-match (car blacklist) path))) (pop blacklist))
    (if blacklist t nil))

      (defun build-cscope-database (root blacklist)
    (setq dirs (list root)) ;; "/usr/include/")) ; I found it to add too much clutter, better add some individual files instead.
    (setq files '())
    (while dirs
      (setq dir (pop dirs))
      (setq ls (directory-files dir))
      (while ls
        (setq file (pop ls))
        (setq file-path (concat dir "/" file))
        (unless (is-path-blacklisted file-path blacklist)
          (if (file-accessible-directory-p file-path)
          (unless (or (string= file ".") (string= file "..")) (add-to-list 'dirs file-path))
        (if (and (file-readable-p file-path) (string-match "\\.\\(c\\|cpp\\|h\\|hpp\\|vala\\|def\\)$" file-path))
            (add-to-list 'files file-path))))))
    ;; Append some commonly used header files. Uncomment the "/usr/include" line above
    ;; and remove/comment this one if you want to see them all their multitude instead
    ;; of this selectivity.
    (dolist (header '("stdio.h" "stdlib.h" "stdarg.h" "ctype.h" "string.h" "math.h" "dlfcn.h"
              "sys/types.h" "sys/time.h" "unistd.h" "assert.h" "limits.h" "poll.h"
              "libgen.h" "signal.h" "fcntl.h"
              "bits/stdio2.h" "bits/string2.h"
              "X11/Xos.h" "X11/Xlib.h" "X11/Xfuncproto.h" "X11/Xatom.h" "X11/Xproto.h"
              "X11/extensions/Xrandr.h" "X11/extensions/shape.h" "X11/cursorfont.h"
              "X11/extensions/XInput2.h" "X11/X.h"
              "boost/bind.hpp" "boost/foreach.hpp"
              "sys/wait.h" "sys/types.h" "sys/stat.h"
              "glib-2.0/glib/gspawn.h" "glib-2.0/glib/gslice.h" "glib-2.0/glib/gmacros.h"
              "glib-2.0/gobject/gobject.h" "glib-2.0/gobject/gtype.h"
              ))
      (add-to-list 'files (concat "/usr/include/" header)))
    files)

      (dump-vars-to-file (build-cscope-database project-root project-blacklist)
             (concat project-root "/" cscope-index-file))

      (message "Processing...")
      (cscope-set-initial-directory project-root)
      (with-temp-buffer (shell-command (concat "cd " project-root "; cscope -b; ctags -eL " cscope-index-file) t)) ; with-temp-buffer is needed to hide output
      (setq tags-file-name (concat project-root "/TAGS"))
      (message "Done processing")))
)

(global-set-key (kbd "<f9>") 'index-project)

Sunday 29 June 2014

Play all songs whose names match a regular expression.


#!/bin/sh
locate -0i --regex "($*).*\.(mp3|mpe?g|avi|mkv|ogg|au|m4a|flac)$" | xargs -0 nohup vlc >/dev/null 2>&1 &

Replace vlc with the player of your choice, save as /usr/local/bin/lplay, make it executable with 'sudo chmod +x /usr/local/bin/lplay' - et voila, you can do pretty complex music queries without amarok or any other bulky and buggy music player, provided there is enough information in their file names.

Since locale uses its own pre-built file database, this will be much faster than using "find" that actually traverses the entire file system.

Examples:
$ lplay music/j-pop
# play your J-Pop collection
$ lplay '風|wind|kaze'
# play everything that has the word or kanji "wind" in it

$ lplay 'bob.*dylan'
# play all Bob Dylan's songs

P.S.
Alternatviely, you can filter out audio/video files by their headers instead of using their extensions. It will give more refined results (you may be surprised how many games keep video and music in standard formats with non-standard extensions!), but will be slower, especially for general queries that have a lot of hits that are not music/video files.
locate -0i --regex "$*" | xargs -0 file -0 | grep -Eia 'audio|video' | cut -d '' -f1 | xargs -d\\n nohup vlc >/dev/null 2>&1 &