bash - dpkg apt

PACKAGE

package states

not-installed # The package is not installed on your system
config-files # Only the configuration files of the package exist on the system
half-installed # The installation of the package has been started, but not completed for some reason
unpacked # The package is unpacked, but not configured
half-configured # The package is unpacked and configuration has been started, but not yet completed for some reason
triggers-awaited # The package awaits trigger processing by another package
triggers-pending # The package has been triggered
installed # The package is correctly unpacked and configured

package selection states

install # The package is selected for installation
hold # A package marked to be on hold is not handled by dpkg, unless forced to do that with option --force-hold
deinstall # The package is selected for deinstallation (i.e. we want to remove all files, except configuration files)
purge # The package is selected to be purged (i.e. we want to remove everything from system directories, even configuration files)

package flags

ok # A package marked ok is in a known state, but might need further processing
reinstreq # A package marked reinstreq is broken and requires reinstallation. These packages cannot be removed, unless forced with option --force-remove-reinstreq

ACTIONS

-i, --install package-file... # Install the package. If --recursive or -R option is specified, package-file must refer to a directory instead
--unpack package-file... # Unpack the package, but don't configure it. If --recursive or -R option is specified, package-file must refer to a directory instead
--configure package...|-a|--pending # Configure a package which has been unpacked but not yet configured.  If -a or --pending is given instead of package, all unpacked but unconfigured packages are configured
--triggers-only package...|-a|--pending # Processes only triggers
-r, --remove package...|-a|--pending # Remove  an  installed  package
-V, --verify [package-name...] # Verifies the integrity of package-name or all packages if omitted, by comparing information from the files installed by a package with the files metadata information stored in the dpkg database
-C, --audit [package-name...] # Performs  database  sanity  and consistency checks for package-name or all packages if omitted (per package checks
--update-avail [Packages-file] # Update dpkg's & old  information  is  replaced  with  the information  in  the  Packages-file
--merge-avail [Packages-file] # Update dpkg's & old information is combined with information from Packages-file
-A, --record-avail package-file... # Update dpkg and dselect's idea of which packages are available with information from the package package-file
--clear-avail # Erase the existing information about what packages are available
--get-selections [package-name-pattern...] # Get list of package selections, and write it to stdout
--set-selections # Set package selections using file read from stdin
--clear-selections # Set the requested state of every non-essential package to deinstall
--yet-to-unpack # Searches for packages selected for installation, but which for some reason still haven't been installed
--predep-package # Print a single package which is the target of one or more relevant pre-dependencies and has itself no unsatisfied pre-dependencies
--add-architecture architecture # Add  architecture  to  the  list of architectures for which packages can be installed without using --force-architecture
--remove-architecture architecture # Remove architecture from the list of architectures for which packages can be installed without using --force-architecture
--print-architecture # Print architecture of packages dpkg installs
--print-foreign-architectures # Print a newline-separated list of the extra architectures dpkg is configured to allow packages to be installed for
--assert-feature # Asserts  that  dpkg  supports  the  requested  feature.     assertable features is:
    support-predepends # Supports the Pre-Depends field
    working-epoch # Supports epochs in version strings
    long-filenames # Supports long filenames in deb(5) archives
    multi-conrep # Supports multiple Conflicts and Replaces
    multi-arch # Supports multi-arch fields and semantics
    versioned-provides # Supports versioned Provides
--validate-thing string # Validate that the thing string has a correct syntax. validatable things is:
    pkgname # Validates the given package name
    trigname # Validates the given trigger name
    archname # Validates the given architecture name
    version # Validates the given version
--compare-versions ver1 op ver2 # Compare  version  numbers,  where op is a binary operator. dpkg returns true (0) if the specified condition is satisfied, and false (1) otherwise
-?, --help # Display a brief help message
--force-help # Give help about the --force-thing options
-Dh, --debug=help # Give help about debugging options
--version # Display dpkg version information
dpkg-deb actions # See dpkg-deb(1) for more information about the following actions
    -b, --build directory [archive|directory] # Build a deb package
    -c, --contents archive # List contents of a deb package
    -e, --control archive [directory] # Extract control-information from a package
    -x, --extract archive directory # Extract the files contained by package
    -X, --vextract archive directory # Extract and display the filenames contained by a
      package
    -f, --field  archive [control-field...]Display control field(s) of a package
    --ctrl-tarfile archiveOutput the control tar-file contained in a Debian package
    --fsys-tarfile archiveOutput the filesystem tar-file contained by a Debian package
    -I, --info archive [control-file...]Show information about a package
dpkg-query actions # See dpkg-query(1) for more information about the following actions
    -l, --list package-name-pattern... # List packages matching given pattern
    -s, --status package-name... # Report status of specified package
    -L, --listfiles package-name... # List files installed to your system from package-name
    -S, --search filename-search-pattern... # Search for a filename from installed packages
    -p, --print-avail package-name... # Display details about package-name, as found in /var/lib/dpkg/available. Users of APT-based frontends should use apt-cache show package-name instead

OPTIONS

--abort-after=number # Change after how many errors dpkg will abort. The default is 50
-B, --auto-deconfigure # When a package is removed, there is a possibility that another installed package depended on the removed package
-Doctal, --debug=octal # Switch debugging on
--force-things
--no-force-things, --refuse-things # Force or refuse (no-force and refuse mean the same thing) to do some things
--ignore-depends=package,... # Ignore dependency-checking for specified packages
--no-act, --dry-run, --simulate # Do everything which is supposed to be done, but don't write any changes
-R, --recursive # Recursively handle all regular files matching pattern *.deb found at specified directories and all of its subdirectories
-G # Don't install a package if a newer version of the same package is already installed. This is an alias of --refuse-downgrade
--admindir=dir # Change default administrative directory, which contains many files that give information about status of installed or uninstalled packages, etc
--instdir=dir # Change default installation directory which refers to the directory where packages are to be installed
--root=dir # Changing root changes instdir to «dir» and admindir to «dir/var/lib/dpkg»
-O, --selected-only # Only process the packages that are selected for installation
-E, --skip-same-version # Don't install the package if the same version of the package is already installed
--pre-invoke=command
--post-invoke=command # Set an invoke hook command to be run via “sh -c” before or after the dpkg run for the unpack, configure,  install,  triggers-only,  remove,  purge,  add-architecture  and  remove-architecture  dpkg  actions
--path-exclude=glob-pattern
--path-include=glob-pattern # Set glob-pattern as a path filter, either by excluding or re-including previously excluded paths matching the specified patterns during install
--verify-format format-name # Sets the output format for the --verify command
--status-fd n # Send machine-readable package status and progress information to file descriptor n
--status-logger=command # Send machine-readable package status and progress information to the shell command s standard input, to be run via “sh -c”
--log=filename #  # Log status change updates and actions to filename, instead of the default /var/log/dpkg.log
--no-debsig # Do not try to verify package signatures
--no-triggers # Do  not  run  any  triggers  in  this run
--triggers # Cancels a previous --no-triggers

Tue Feb 25 08:51:14 2020 · permalink

http://code.ambau.ovh/snippets/dpkg.html

bash - sed man

MAN

synopsis

sed [OPTION]... {script-only-if-no-other-script} [input-file]...

sed [-options] [commande] [] sed [-n [-e commande] [-f script] [-i[.extension]] [l [cesure]] rsu] [] []

[adresse[,adresse]][!]commande[arguments]

[adresse[,adresse]]{
 commande1
 commande2;commande3
}

options

-n, --quiet, --silent # suppress automatic printing of pattern space
-e script, --expression=script # add the script to the commands to be executed
-f script-file, --file=script-file # add the contents of script-file to the commands to be executed
--follow-symlinks # follow symlinks when processing in place
-i[SUFFIX], --in-place[=SUFFIX] # edit files in place (makes backup if SUFFIX supplied)
-l N, --line-length=N # specify the desired line-wrap length for the `l' command
--posix # disable all GNU extensions
-E, -r, --regexp-extended # use extended regular expressions in the script (for portability use POSIX -E)
-s, --separate # consider files as separate rather than as a single, continuous long stream
--sandbox # operate in sandbox mode
-u, --unbuffered # load minimal amounts of data from the input files and flush the output buffers more often
-z, --null-data # separate lines by NUL characters
--help # display this help and exit
--version # output version information and exit

Zero-address ``commands''

: label # Label for b and t commands
'#'comment # The comment extends until the next newline (or the end of a -e script fragment)
} # The closing bracket of a { } block

Zero- or One- address commands

= # Print the current line number
a \text # Append text, which has each embedded newline preceded by a backslash
i \text # Insert text, which has each embedded newline preceded by a backslash
c \text # Replace text, which has each embedded newline preceded by a backslash
q [exit-code] # Immediately quit the sed script without processing any more input, except that if auto-print is not disabled the current pattern space will be printed
Q [exit-code] # Immediately quit the sed script without processing any more input
r filename # Append text read from filename
R filename # Append a line read from filename. Each invocation of the command reads a line from the file. This is a GNU extension

Commands which accept address ranges

{ # Begin a block of commands (end with a })
b label # Branch to label; if label is omitted, branch to end of script
c \text # Replace the selected lines with text, which has each embedded newline preceded by a backslash
d # Delete pattern space. Start next cycle
D # If pattern space contains no newline, start a normal new cycle as if the d command was issued. Otherwise, delete text in the pattern space up to the first newline, and restart cycle with the resultant pattern space, without reading a new line of input
h H # Copy/append pattern space to hold space
g G # Copy/append hold space to pattern space
l # List out the current line in a ``visually unambiguous'' form
l width # List out the current line in a ``visually unambiguous'' form, breaking it at width characters. This is a GNU extension
n N # Read/append the next line of input into the pattern space
p # Print the current pattern space
P # Print up to the first embedded newline of the current pattern space
s/regexp/replacement/ # Attempt to match regexp against the pattern space. If successful, replace that portion matched with replacement. The replacement may contain the special character & to refer to that portion of the pattern space which matched, and the special escapes \1 through \9 to refer to the corresponding matching sub-expressions in the regexp
t label # If a s/// has done a successful substitution since the last input line was read and since the last t or T command, then branch to label; if label is omitted, branch to end of script
T label # If no s/// has done a successful substitution since the last input line was read and since the last t or T command, then branch to label; if label is omitted, branch to end of script. This is a GNU extension
w filename # Write the current pattern space to filename
W filename # Write the first line of the current pattern space to filename. This is a GNU extension
x # Exchange the contents of the hold and pattern spaces
y/source/dest/ # Transliterate the characters in the pattern space which appear in source to the corresponding character in dest

Addresses

Sed commands can be given with no addresses, in which case the command will be executed for all input lines; with one address, in which case the command will only be executed for input lines which match that address; or with two addresses, in which case the command will be executed for all input lines which match the inclusive range of lines starting from the first address and continuing to the second address. Three things to note about address ranges: the syntax is addr1,addr2 (i.e., the addresses are separated by a comma); the line which addr1 matched will always be accepted, even if addr2 selects an earlier line; and if addr2 is a regexp, it will not be tested against the line that addr1 matched

After the address (or address-range), and before the command, a ! may be inserted, which specifies that the command shall only be executed if the address (or address-range) does not match

The following address types are supported:

number # Match only the specified line number (which increments cumulatively across files, unless the -s option is specified on the command line)
first~step # Match every step'th line starting with line first. For example, ``sed -n 1~2p'' will print all the odd-numbered lines in the input stream, and the address 2~5 will match every fifth line, starting with the second. first can be zero; in this case, sed operates as if it were equal to step. (This is an extension.)
$ # Match the last line
/regexp/ # Match lines matching the regular expression regexp
\cregexpc # Match lines matching the regular expression regexp. The c may be any character

GNU sed also supports some special 2-address forms:

0,addr2 # Start out in "matched first address" state, until addr2 is found. This is similar to 1,addr2, except that if addr2 matches the very first line of input the 0,addr2 form will be at the end of its range, whereas the 1,addr2 form will still be at the beginning of its range. This works only when addr2 is a regular expression
addr1,+N # Will match addr1 and the N lines following addr1
addr1,~N # Will match addr1 and the lines following addr1 until the next line whose input line number is a multiple of N

bash · bashroot

Tue Feb 25 07:47:38 2020 * · permalink

http://code.ambau.ovh/snippets/sed-man.html

markdown

base

italic italic

bold bold

Titre1

Titre2

Titre6

listA
- listB
- listB
listA

list1
1. list2
list1

Une citation

Une réponse à la citation

Puce

Autre puce

code1

code2

URL alias

image alias

separator line

Name	Lunch order	Spicy	Owes
Joan	saag paneer	medium	$11
Sally	vindaloo	mild	$14
Erin	lamb madras	HOT	$5

There are multiple syntax highlighting themes to choose from. Here s one of them:

// All the code you will ever need
var hw = "Hello World!"
alert(hw);

--- comment
SELECT tat FROM mybase WHERE id.tat IS LIKE '%TATA%';

DROP DATABASE test;

My math is so rusty that I barely remember the quadratic equation:
$-b \pm \sqrt{b^2 - 4ac} \over 2a$

comment

To improve platform compatibility

Wed Feb 19 16:06:49 2020 * · permalink

http://code.ambau.ovh/snippets/markdown.html

gpg

https://doc.ubuntu-fr.org/gnupg

CONF

~/.gnupg/gpg.conf

# default key to use
default-key <key-uid>

LIST

--list-keys # list all keys
--list-keys --keyid-format {none|short|0xshort|long|0xlong} # list keys with a specified format
--list-secret-keys # list only secret keys
--list-public-keys # list only public keys

--search-keys <id>|<identifier>|<email> # search a key with id, identifier, email ...

KEY

--full-gen-key # generate a key (rsa, 4096, 2y)
--delete-keys <id> # delete keys with id
--gen-revoke <email> # generate a certificate of revocation. keep it safely !
--fingerprint <id> # verify the fingerprint of key
--sign-key <id> # sign a trsuted public key

SERVER

--send-key <id> [--keyserver <serveur>] # send key to specified server or the default one
--recv-keys <id>  [--keyserver <serveur>]  # receive key with id from server

FILE

--verify <file.sign> <file> # verify the file are the same hash than the signed reference
--clearsign <file> # sign a file

# the file must be encrypted with the recipient's public key
# --sign & --encrypt can be use together
gpg --encrypt <file> # encrypt file but decrypted stay in binary mode
gpg --armor --output "file.gpg" --encrypt "file" # encrypt file and decrypted are available for reading
gpg --output <file> --decrypt <file.gpg> # decrypt file

bash

Sun Feb 16 21:40:21 2020 * · permalink

http://code.ambau.ovh/snippets/gpg.html

lxc

chapter
CONFIG
COPY
DNSMASQ
LIST
IMAGE
INIT
NETWORK
PROFILE
STORAGE
EXEC
PUBLISH
TRICK

CONFIG

^

set network configuration

echo -e "lxc.network.0.ipv4 = 10.100.0.10/24\nlxc.network.0.ipv4.gateway = 10.100.0.1\n" | lxc config set my-container raw.lxc -

COPY

^

Copy container with changing device key/value

lxc copy $ct $ctnew --device $devicename,$key=$value
# lxc copy srv-mail srv-mail-maria --device eth0,ipv4.address=10.0.0.10

DNSMASQ

^

# /path/to/host-to-ip-file.conf with following dnsmasq syntax
c1,10.100.0.10
c2,10.100.0.20
&
lxc network set lxdbr0 set raw.dnsmasq hostsfile=/path/to/host-to-ip-file.conf
lxc restart c1 c2

LIST

^

lxc list $str # be careful: list container names with a name contains $str
lxc list $regexp # filters containers list with name matches regexp
lxc list $property_name=$value # filter container list with properties matches values (use list --format json to see real properties name)
lxc list -c 46abcdlnNpPsSt L # show all container informations
lxc list -c n4s # outputs minimal container informations
lxc list image.os=Alpine image.architecture=amd64 # outputs informations for containers matched with properties: image.os=Alpine image.architecture=amd64

# list names of all containers
lxc list -f csv -c n
lxc list --format=json | jq -r '.[].name
# list names of running containers
lxc list status=Running -f csv -c n
lxc list --format=json | jq -r '.[] | select(.status == "Running").name
# list names of stopped containers
lxc list status=Stopped -f csv -c n
lxc list --format=json | jq -r '.[] | select(.status == "Stopped").name
# list image with finding any of aliases
lxc image list --format json | jq -r '.[].aliases[] | select(.name == "debian10").name'

IMAGE

^

lxc image list $repo:$os/$release
lxc image list $repo: architecture=$regexp description=$regexp os=$regexp release=$regexp serial=$regexp

lxc image list ubuntu: # list availables images from repository ubuntu
lxc image list images:debian # list availables images from repository images started debian"
lxc image list images:alpine/3.11/amd64 # list availables images from repository images matched with os:Alpine release:3.11 architecture:amd64
lxc image list images: architecture=amd64 os=Debian release=buster

lxc image copy images:debian/10/amd64 local: --alias debian10 --alias debian-10 --alias buster --auto-update
lxc image copy images:alpine/3.12 local: --alias alpine312 --alias alpine-312 --auto-update
lxc image copy ubuntu:18.04/amd64 local: --alias bionic --alias ubuntu1804 --alias ubuntu-1804 --auto-update
lxc image copy ubuntu:20.04/amd64 local: --alias focal --alias ubuntu2004 --alias ubuntu-2004 --auto-update
lxc image copy images:centos/8/amd64 local: --alias centos8 --alias centos-8 --auto-update

INIT

^

lxc init alpine311 alpine311 # Initialize image with name

NETWORK

^

lxc network set $inet_name ipv4.address $address_cidr # set network address for interface $inet
lxc network create $inet_name ipv4.address=10.0.1.1/24 # add network interface
lxc network attach-profile $inet_name stock eth0

PROFILE

^

lxc profile device add prod root disk pool=prod path=/ # add root path to profile from a pool 'prod'
lxc profile device add $profile_name $device_name disk pool=$pool_name path=$path_ct # add $device_name to $profile_name for $path_ct
lxc profile device add $profile_name $device_name disk source=$path_host path=$path_ct

STORAGE

^

lxc storage create default zfs size=50GB # create a pool using a loop device named 'default' with zfs driver & a size of 50G
lxc storage create stock zfs source=stock # create a pool 'stock' using a zfs pool named 'stock'

EXEC

^

echo -e "auth-zone=lxd\ndns-loop-detect" | lxc network set lxdbr0 raw.dnsmasq -

PUBLISH

^

lxc publish $CTNAME --alias $CTNAME-$HOSTNAME # default compression are gzip
lxc publish $CTNAME --alias $CTNAME-$HOSTNAME --compression xz # good compression but long

TRICK

^

pretty print

json

lxc list --format=json $ctname$ # '$' is to limit to exact name & not only started with ctname

# pretty print
lxc list --format=json $ctname$ | jq
lxc list --format=json $ctname$ | python -m json.tool

yaml

lxc list --format=yaml $ctname$ # '$' is to limit to exact name & not only started with ctname

# pretty print
lxc list --format=yaml $ctname$ |yq r - -C

profile

yaml

Print name of host interfaces attached to the profile $profile

lxc profile show $profile | yq r - "devices.(type==nic).parent"

Print name of containers that use the profile $profile

lxc profile show $profile | yq r - 'used_by' | sed 's|^.*/||'

bash · lxc · lxd

Wed Feb 5 08:16:13 2020 * · permalink

http://code.ambau.ovh/snippets/lxc.md

docker

LXC

# permit to a container to run in lxc container without 'lxc.apparmor.profile: unconfined' or

security.nesting = true

bash · docker

Mon Feb 3 21:41:03 2020 * · permalink

http://code.ambau.ovh/snippets/docker.html

alpine

chapter
OPTIONS
ADD
DEL
System maintenance
FIX
UPDATE
UPGRADE
CACHE
Querying information about packages
INFO
LIST
SEARCH
STATS
DOT
POLICY
Repository maintenance
INDEX
FETCH
VERIFY
MANIFEST
RC
RC-STATUS
RC-UPDATE
RC-SERVICE

OPTIONS

^

Global options

  -h, --help # Show generic help or applet specific help
  -p, --root DIR # Install packages to DIR
  -X, --repository REPO # Use packages from REPO
  -q, --quiet # Print less information
  -v, --verbose # Print more information (can be doubled)
  -i, --interactive # Ask confirmation for certain operations
  -V, --version # Print program version and exit
  -f, --force # Enable selected --force-* (deprecated)
  --force-binary-stdout # Continue even if binary data is to be output
  --force-broken-world # Continue even if 'world' cannot be satisfied
  --force-non-repository # Continue even if packages may be lost on reboot
  --force-old-apk # Continue even if packages use unsupported features
  --force-overwrite # Overwrite files in other packages
  --force-refresh # Do not use cached files (local or from proxy)
  -U, --update-cache # Alias for --cache-max-age 1
  --progress # Show a progress bar
  --progress-fd FD # Write progress to fd
  --no-progress # Disable progress bar even for TTYs
  --purge # Delete also modified configuration files (pkg removal) and uninstalled packages from cache (cache clean)
  --allow-untrusted # Install packages with untrusted signature or no signature
  --wait TIME # Wait for TIME seconds to get an exclusive repository lock before failing
  --keys-dir KEYSDIR # Override directory of trusted keys
  --repositories-file # REPOFILE Override repositories file
  --no-network # Do not use network (cache is still used)
  --no-cache # Do not use any local cache path
  --cache-dir CACHEDIR # Override cache directory
  --cache-max-age AGE # Maximum AGE (in minutes) for index in cache before refresh
  --arch ARCH # Use architecture with --root
  --print-arch # Print default arch and exit

Commit options

  -s, --simulate # Show what would be done without actually doing it
  --clean-protected # Do not create .apk-new files in configuration dirs
  --overlay-from-stdin # Read list of overlay files from stdin
  --no-scripts # Do not execute any scripts
  --no-commit-hooks # Skip pre/post hook scripts (but not other scripts)
  --initramfs-diskless-boot # Enables options for diskless initramfs boot

ADD

^

Add PACKAGEs to 'world' and install (or upgrade) them, while ensuring that all dependencies are met

apk add [OPTIONS...] PACKAGE...

  --initdb # Initialize database
  -u, --upgrade # Prefer to upgrade package
  -l, --latest # Select latest version of package (if it is not pinned), and print error if it cannot be installed due to other dependencies
  -t, --virtual NAME # Instead of adding all the packages to 'world', create a new virtual package with the listed dependencies and add that to 'world'; the actions of the command are easily reverted by deleting the virtual package

examples

apk add $pkg=$version # install and fix package $pkg in version $version. ex: apk add bash=5.0.0-r0
apk add $pkg=~$major_version # install and fix package $pkg in major version $major_version. ex: apk add bash=~5.0
apk add $pkg=$version # install version $version of package $pkg & remove version holding. ex: apk add bash>5.0.0-r0
apk add --allow-untrusted $path/$pkg.apk # install untrusted package from file

DEL

^

Remove PACKAGEs from 'world' and uninstall them

apk del [OPTIONS...] PACKAGE...

  -r, --rdepends # Recursively delete all top-level reverse dependencies too

FIX

^

Repair package or upgrade it without modifying main dependencies

apk fix [OPTIONS...] PACKAGE...

  -d, --depends # Fix all dependencies too
  -r, --reinstall # Reinstall the package (default)
  -u, --upgrade # Prefer to upgrade package
  -x, --xattr # Fix packages with broken xattrs
  --directory-permissions # Reset all directory permissions

UPDATE

^

Update repository indexes from all remote repositories

apk update

UPGRADE

^

Upgrade currently installed packages to match repositories

apk fix [OPTIONS...]

  -a, --available # Resets versioned world dependencies, and changes to prefer replacing or downgrading packages (instead of holding them) if the currently installed package is no longer available from any repository
  -l, --latest # Select latest version of package (if it is not pinned), and print error if it cannot be installed due to other dependencies
  --no-self-upgrade # Do not do early upgrade of 'apk-tools' package
  --self-upgrade-only # Only do self-upgrade

CACHE

^

Download missing PACKAGEs to cache and/or delete unneeded files from cache

apk cache [OPTIONS...] sync | clean | download

  -u, --upgrade # Prefer to upgrade package
  -l, --latest # Select latest version of package (if it is not pinned), and print error if it cannot be installed due to other dependencies

examples

apk cache -v sync # clean cache & download missing packages

INFO

^

Give detailed information about PACKAGEs or repositories

apk info [OPTIONS...] PACKAGE...

  -L, --contents # List contents of the PACKAGE
  -e, --installed # Check if PACKAGE is installed
  -W, --who-owns # Print the package owning the specified file
  -R, --depends # List packages that the PACKAGE depends on
  -P, --provides # List virtual packages provided by PACKAGE
  -r, --rdepends # List all packages depending on PACKAGE
  --replaces # List packages whom files PACKAGE might replace
  -i, --install-if # List the PACKAGE's install_if rule
  -I, --rinstall-if # List all packages having install_if referencing PACKAGE
  -w, --webpage # Show URL for more information about PACKAGE
  -s, --size # Show installed size of PACKAGE
  -d, --description # Print description for PACKAGE
  --license # Print license for PACKAGE
  -t, --triggers # Print active triggers of PACKAGE
  -a, --all # Print all information about PACKAGE

LIST

^

List packages by PATTERN and other criteria

apk list [OPTIONS...] PATTERN

  -I, --installed # List installed packages only
  -O, --orphaned # List orphaned packages only
  -a, --available # List available packages only
  -u, --upgradable # List upgradable packages only
  -o, --origin # List packages by origin
  -d, --depends # List packages by dependency
  -P, --providers # List packages by provider

SEARCH

^

Search package by PATTERNs or by indexed dependencies

apk search [OPTIONS...] PATTERN

  -a, --all # Show all package versions (instead of latest only)
  -d, --description # Search package descriptions (implies -a)
  -x, --exact # Require exact match (instead of substring match)
  -e # Synonym for -x (deprecated)
  -o, --origin # Print origin package name instead of the subpackage
  -r, --rdepends # Print reverse dependencies of package
  --has-origin # List packages that have the given origin

STATS

^

Show statistics about repositories and installations

apk stats

examples

apk stats # Show statistics about repositories and installations

DOT

^

Generate graphviz graphs

apk dot [OPTIONS...] PKGMASK...

  --errors # Output only parts of the graph which are considered erroneous: e.g. cycles and missing packages
  --installed # Consider only installed packages

POLICY

^

Show repository policy for packages

apk policy

INDEX

^

Create repository index file from FILEs

apk index [OPTIONS...] FILE...

  -o, --output FILE # Write the generated index to FILE
  -x, --index INDEX # Read INDEX to speed up new index creation by reusing the information from an old index
  -d, --description TEXT # Embed TEXT as description and version information of the repository index
  --rewrite-arch ARCH # Use ARCH as architecture for all packages

FETCH

^

Download PACKAGEs from global repositories to a local directory

apk fetch [OPTIONS...] PACKAGE...

  -L, --link # Create hard links if possible
  -R, --recursive # Fetch the PACKAGE and all its dependencies
  --simulate # Show what would be done without actually doing it
  -s, --stdout # Dump the .apk to stdout (incompatible with -o, -R, --progress)
  -o, --output DIR # Directory to place the PACKAGEs to

VERIFY

^

Verify package integrity and signature

apk verify FILE...

MANIFEST

^

Show checksums of package contents

apk manifest PACKAGE...

RC-STATUS

^

run-level

boot – Generally the only services you should add to the boot runlevel are those which deal with the mounting of filesystems, set the initial state of attached peripherals and logging. Hotplugged services are added to the boot runlevel by the system. All services in the boot and sysinit runlevels are automatically included in all other runlevels except for those listed here
single – Stops all services except for those in the sysinit runlevel
reboot – Changes to the shutdown runlevel and then reboots the host
shutdown – Changes to the shutdown runlevel and then halts the host
default – Used if no runlevel is specified. (This is generally the runlevel you want to add services to.)

rc-status

rc-status # show services attached to actual run-level
rc-status boot # show services attached to run-level 'boot'
-l --list # Show list of run levels
-a --all # Show services from all run levels
-s --servicelist # Show service list
-m, --manual # show manually started services
-u, --unused # show services not assigned to any runlevel
-c, --crashed # show crashed services
-S, --supervised # show supervised services

RC-UPDATE

^

rc-update

rc-update show {run-level} # show services attached to a run-level
rc-update add {service} {run-level} # attach service to a run-level
rc-update del {service} {run-level} # dettach service to a run-level

rc

rc $runlevel # change run-level to $runlevel

RC-SERVICE

^

rc-service

rc-service {service} start # start service
rc-service {service} stop # stop service
rc-service {service} restart # restart service
-e, --exists <arg> # tests if the service exists or not
-c, --ifcrashed # if the service is crashed run the command
-i, --ifexists # if the service exists run the command
-I, --ifinactive # if the service is inactive run the command
-N, --ifnotstarted # if the service is not started run the command
-s, --ifstarted # if the service is started run the command
-S, --ifstopped # if the service is stopped run the command
-l, --list # list all available services
-r, --resolve <arg> # resolve the service name to an init script
-Z, --dry-run # dry run (show what would happen)
-q, --quiet # run quietly (repeat to suppress errors)

alpine · bash · linux

Sat Feb 1 16:34:48 2020 * · permalink

http://code.ambau.ovh/snippets/alpine.html

yt-dlp

yt-dlp $link # download $link with better video & audio format
 -F # list all formats available for $link
 -f $format # download $link with the specified formats (audio or video, separate files)
 -f $videoformat+$audioformat --merge-output-format # download $link with the specified formats & merge it to a file. One of mkv, mp4,ogg, webm, flv
 -o $file # specify the output file name
 --audio-format $format # specify audio format: "best", "aac", "flac", "mp3", "m4a", "opus", "vorbis", or "wav";  "best"  by  default; No effect without -x

FORMAT

(mp4,webm)[height<480] # use first mp4, second webm with a maximum height of 480

examples

yt-dlp $link -f 'webm[height<800]+bestaudio' --merge-output-format webm # merge specified video + audio formats
yt-dlp $link -f '243+251' --merge-output-format webm # merge specified video + audio formats

manual rip the videos list
After copying right div with inspector from youtube.com in file videos_list.html

videos_file="videos_list.html"
videos_format="244+250"
videos_id=`sed -n "s|.*watch?v=\([^&]\+\)&.*|\1|p" "${videos_file}" | uniq | xargs`
for id in ${videos_id}; do echo yt-dlp "https://www.youtube.com/watch?v=${id}" -f "${videos_format}"; done

create batch from youtube list

grep 'watch?v=' ~/Downloads/yt_ls |sed "s|.*watch?v=\([^&]\+\)&.*|\1|"| sort -u > yt_id
format="247+251"
yt-dlp $(head -n1 yt_id) -F
is=$(wc -l < yt_id); i=0; while read id; do i=$((i+1)); echo "----- $i / $is"; yt-dlp $id -f ${format} || echo $id >> yt_err; done < yt_id

bash · youtube · yt-dlp

Tue Jan 14 07:37:55 2020 * · permalink

http://code.ambau.ovh/snippets/yt-dlp.html

vim

https://github.com/junegunn/vim-plug

chapter	designation
NORMAL MODE	For navigation and manipulation of text. This is the mode that vim will usually start in
COMMAND MODE	For executing extra commands like the help, shell, ...
INSERT/EDIT MODE	Press i to enter insert/edit/mode & q or <esc> to quit

3 main modes

Normal mode

For navigation and manipulation of text. This is the mode that vim will usually start in

Command mode

For executing extra commands like the help, shell, ...

Insert (edit) mode

For inserting new text, where you type into your file like other editors.

NORMAL MODE

^

MOTIONS

move around the text (file) by

                   gg
                 ctrl+b
                 ctrl+u
                    -
                    k
                    ^
0 / ^ / B / b / h <   > l / e / E / w / W / $
                    v
                    +
                    j
                 ctrl+d
                 ctrl+f
                    G

arrow keys

    k
    ^
h <   >l
    j

word

w # next word
W # next WORD
b # previous word
B # previous WORD
e # end of word
E # end of WORD

w / W # word / WORD
[, ] # block
(, ) # block
<, > # block
", ' #" in double quote or quote
t # XML/HTML tag
s # sentence

line

0 # begin of line (column 0)
^ # begin of line (non-blank)
$ # end of line
\- = k^ # start of previous line
\+ = j^ # start of next line

file

gg / G # go to begin / end of file
[num]gg / [num]G / :num<CR>  # go to line num
gd # go to definition of current word
gf # go to the file (under the cursor)

EDITING

syntax

set ft=prolog # set the file type to prolog

copy/paste:

yy # yank/copy current line
p # paste to the next line
P # paste above current line

commands

. # repeat last command
~ # swap case
x # delete char current char
r # replace char under the cursor
J # merge with the next line
dd # delete current line
D / C # delete line from cursor to the EOL
u # undo
ctrl+r # redo

visual mode

v # into visual/select mode
V # into visual/select mode by line
ctrl+v # into visual/select mode by block

alignment

== # auto indent
>> # shift right, increase indent
<< # shift left, decrease indent

examples

di) # delete the text inside current paranthesis
ci" #" change the text inside ""
gUiw # make the word under the cursor to upper case

registers

"[char][y|d] #" yank/delete into register
"[char][p|P] #" paste from register
:echo @[char] # shows register content
:reg[isters] # shows all registers

macro

q[char] # start recording into register char
q # stop recording macro
@[char] # play macro in register char
@@ # repeat last playback

code folding

zi # toggles folding on or off
za # toggles current fold open or close
zc # close current fold
zC # close current fold recursively
zo # open current fold
zO # open current fold recursively
zR # open all folds
zM # close all folds
zv # expand folds to reveal the cursor
zk / zj # move to previous / next fold

WINDOW

move inside

H # top of window
M # middle of window
L # low (bottom) of window
zt # scroll to top
zz # scroll to middle
zb # scroll to bottom
ctrl+b / ctrl+f # previous / next page
ctrl+u / ctrl+d # previous / next half page

split

ctrl+w s = :sp[lit] # split current window horizontally
ctrl+w v = :vs[plit] # split current window vertically
ctrl+w c = :cl[ose] # close current window
ctrl+w o = :on[ly] # close all windows except current one
ctrl+w ctrl+w # switch to next split window
ctrl+w ctrl+p # switch to previous split window
ctrl+w hjkl # switch (move cursor) to left, below, above or right split
ctrl+w HJKL # move current window to left, below, above or right
ctrl+w r # rotate window clockwise
ctrl+w = # make all windows equal in size
[num]ctrl+w +- # increase/decrease current window height
[num]ctrl+w <> # increase/decrease current window width
ctrl+w _ # minimize current window
ctrl+w T # move current window to new tab

JUMPS & MARKS

ctrl+o # jump/switch back in the buffer history
ctrl+i # jump/switch forward in the buffer history
ctrl+6 # jump/switch to the buffer you just left
ctrl+] # jump/switch to tag under cursor (if ./tags is available)
' ' # jump/switch back to last jump
'. #' jump/switch to last edited line

} # next paragraph
{ # previous paragraph

% # switch matching (), {} or []
m[char] / '[char] # mark by / jump to [char]
m[CHAR] / '[CHAR] # mark by / jump to [CHAR] across the files.

SPELL CHECKING

]s # jump to next spelling error
[s # jump to previous spelling error
z= # suggest corrections for current word
zg # add current word to the dictionary
zw # remove current word from dictionary

SEARCHING

word

- # find next current word
'#' # find previous current word
/[pattern] # search forward by matching pattern
?[pattern] # search backward by matching pattern
n # next result
N # previous result

[I # show lines with matching word under cursor

character

f[char] # find next exact character in the line
F[char] # find previous exact character in the line
t[char] # find next exact character in the word
T[char] # find previous exact character in the line
; next repeat for f/t/F/T action
, previous repeat for f/t/F/T action

COMMAND MODE

^

editing the text without transition to Insert Mode:

@: # repeat last command-line change (command invoked with ":", for example :s/old/new/).

windows and splits

:sp[lit] = ctrl+w s # split current window horizontally
:vs[plit] = ctrl+w v # split current window vertically
:cl[ose] = ctrl+w c # close current window
:on[ly] = ctrl+w o # close all windows except current one

lists

:jumps # shows the jump list
:changes # shows the change list
:reg[isters] # shows the registers
:marks # shows the marks
:delm[arks] {marks} # delete specified mark(s)
  delm a b 1 \" # deletes a, b, 1 and "
  delm a-h # deletes all marks from a to h
:delm[marks]! # deletes all lowercase marks

file and buffers

:w[rite] # write current file
:q # close/quit current file, split or tab
:wq = ZZ # write current file and quit
:q! = ZQ # quit without writing the changes
:qa # quit all splits
:ls # list all open files/buffers
:f[ile] = ctrl+g # shows current file path
:e[dit] # open a file for editing
:ene[w] # open a blank new file for editing
:b<n> # jump to buffer n returned from :ls
:b<file> # jump to buffer file, Tab to scroll through available options
:bn[ext] # jump to next buffer
:bp[rev] # jump to previous buffer
:bd[elete] # remove file from buffer list

shell

:mak[e] # run make in current directory
:cw # toggle mini window for errors
:! # executes external shell command
:r[ead] # read external program output into current file

tabs

ctrl+w gf # open file under the cursor into new tab
:tabs # list current tabs and windows
:tabn = <ctrl+PageDown> # next tab
:tabn <n> # goto tab n
:tabp = tabN = <ctrl+PageUp> # previous tab
:tabe [file] # create a new blank tab or opens file in that tab

OPERATORS

operator are generally constructed as:
[operator][count][motion]
[operator]i[motion]

operators:

c # change command ...
d # delete ...
y # yank (copy) ...
g~ # swap case ...
gu # to lower case ...
gU # to upper case ...

HELP

:h cmd # normal mode command help
:h :cmd # command line help for cmd
:h i_cmd # insert mode command help
:h v_cmd # visual mode command help
:h c_cmd # command line editing cmd help
:h 'option' # help of option
:helpg[rep] # search through help docs!

special help sections

:h motions
:h word-motions
:h jump-motions
:h mark-motions
:h operators
:h buffres
:h windows
:h tabs
:h registers
:h pattern-searches

OPTIONS

:set <opt>? # shows current option value
:set no<opt> # turn off flag opt
:set opt # turn on flag opt
:set opt=val # override value of opt
:set opt+=val # append val to opt
:echo &opt # shows value of opt

essential options

hidden or hid # when off, a buffer is unloaded when it's abandoned.
laststatus or ls # shows status line # 0 (never), 1 (only if at least two windows), 3 (always)
hisearch or his # highlight search matches
number or nu # shows line number
showcmd or sc # shows command as you type them (may not be available on your compilation)
ruler or ru # shows line and column number of the cursor
wrap # controls line wrapping
ignorecase or ic # ignores case for search patterns
smartindent or si # flag for smart indenting
foldmethod or fdm # fold method
spell / nospell # turn spell checking enable or disable.

SUBSTITUTE

:s/search/replace/ # basic substitution on a line
:%s/search/replace/ # run substitution on every line
:%s/search/replace/g # g flag means apply to every match
:%s/search/replace/c # c flag means ask for confirmation

tags / ctags

by executing $> ctags -r under project tree:

:tag <name>TAB # goes to tag name
ctrl+] # goes to the tag under cursor

INSERT/EDIT MODE

^

insert

i # insert at left of cursor
a # insert at right of cursor
I # insert at the line beginning (non-blank)
A # insert at end of line
o # insert by adding new line below the cursor
O # insert by insert new line above the cursor
s # substitute at cursor and enter insert mode
S = ^DA = ddO # delete current line and enter insert mode
C = c$ # change line from cursor to EOL

mode change

Esc = ctrl+c = ctrl+[ # exit insert mode

auto complete

ctrl+p # auto-complete / previous item
ctrl+n # auto-complete / next item
ctrl+xctrl+l # auto complete line mode

cool editing stuff

ctrl+w # delete word before cursor
ctrl+u # delete line before cursor
ctrl+r[char] # insert content of register [char]
ctrl+t # increase line indent
ctrl+u # decrease line indent

bash · vi

Fri Dec 20 00:58:19 2019 * · permalink

http://code.ambau.ovh/snippets/vim.html

vi - vim - vimdiff

https://gist.github.com/azadkuh/5d223d46a8c269dadfe4

OPTIONS

:syntax on # enable syntax highlightning
:syntax off # disable syntax highlightning

:set nu / :set number # show line numbers
:set nonu / :set nonumber / set nu! # hide line numbers

VIMDIFF

ctrl+(w+w) # toggle buffer

do # get changes from other window into the current window
dp # put the changes from current window into the other window
]c # jump to the next change
[c # jump to the previous change
zo # open fold
zc # close fold
zr # reducing folding level
zm # one more folding level, please
:diffupdate, :diffu # recalculate the diff
:diffg RE # get from REMOTE
:diffg BA # get from BASE
:diffg LO # get from LOCAL

VIM to VIMDIFF

:vs file # vertical split with file
:split file # horizontal split file
ctrl+w ctrl+w # switch cursors to different split screen
:diffthis # invoke "diff mode" in file
:diffthis # switch to other file and invoke "diff mode"
:diffoff # turn off "diff mode"

bash · vi

Thu Dec 19 11:22:48 2019 * · permalink

http://code.ambau.ovh/snippets/vi.html

debian 10 - buster

SSH

change identity of key in 'authorized_keys'

file="/root/.ssh/authorized_keys"
sudo sed '/manjaro@970g/ s|^.* \(ssh-.*\)$|\1|' $file
sudo systemctl restart sshd.service

APT SOURCES

add contrib to main backports

file="/etc/apt/sources.list"
sed -i '/backports/ s| main| main contrib|' $file
apt update

ZFS

install zfs

apt install zfs-dkms zfsutils-linux # install zfs tools
echo -e "# zfs utils\nzfs" >> /etc/modules
modprobe zfs # or reboot
lsmod | grep zfs # verify zfs module are correctly loaded
systemctl status zfs-* # verify all zfs services are correctkly started

format

fdisk $device # use 36 for FreeBSD type

bash · debian · linux

Sun Dec 15 01:43:13 2019 * · permalink

http://code.ambau.ovh/snippets/debian_10_buster.html

zfs

https://medium.com/@cq94/zfs-vous-connaissez-vous-devriez-1d2611e7dad6

The zfs command configures ZFS datasets within a ZFS storage poo. A dataset is identified by a unique path within the ZFS namespace

A dataset can be one of the following:

File system
A ZFS dataset of type filesystem can be mounted within the standard system namespace and behaves like other file systems
Volume
A logical volume exported as a raw or block device
Snapshot
A read-only version of a file system or volume at a given point in time. It is specified as filesystem@name or volume@name.
Bookmark
Much like a snapshot, but without the hold on on-disk data. It can be used as the source of a send (but not for a receive). It is specified as filesystem#name or volume#name.

ZFS File System Hierarchy

A ZFS storage pool is a logical collection of devices that provide space for datasets. A storage pool is also the root of the ZFS file system hierarchy

Snapshots
A snapshot is a read-only copy of a file system or volume
Bookmarks
A bookmark is like a snapshot, a read-only copy of a file system or volume. Bookmarks can be created extremely quickly, compared to snapshots, and they consume no additional space within the pool. Unlike snapshots, bookmarks can not be accessed through the filesystem in any way
Clones
A clone is a writable volume or file system whose initial contents are the same as another dataset. Clones can only be created from a snapshot. As with snapshots, creating a clone is nearly instantaneous, and initially consumes no additional space
Mount Points
Creating a ZFS file system is a simple operation, so the number of file systems per system is likely to be numerous
Deduplication
Deduplication is the process for removing redundant data at the block level, reducing the total amount of data stored

SUBCOMMANDS

subcommand	Designation
CREATE	Creates a new ZFS file system
DESTROY	Destroys the given dataset
SNAPSHOT	Creates snapshots with the given names
ROLLBACK	Roll back the given dataset to a previous snapshot
CLONE	Creates a clone of the given snapshot
PROMOTE	Promotes a clone file system to no longer be dependent on its "origin" snapshot
RENAME	Renames dataset
LIST	Lists the property information for the given datasets in tabular form
SET	Sets the property or list of properties to the given value(s) for each dataset
GET	Displays properties for the given datasets
INHERIT	Clears the specified property, causing it to be inherited from an ancestor
USERSPACE	Displays space consumed by, and quotas on, each user in the specified filesystem or snapshot
GROUPSPACE	Displays space consumed by, and quotas on, each group in the specified filesystem or snapshot
MOUNT	Displays all ZFS file systems currently mounted or mounts it
UNMOUNT	Unmounts currently mounted ZFS file systems
SHARE	Shares available ZFS file systems
UNSHARE	Unshares currently shared ZFS file systems
BOOKMARK	Creates a bookmark of the given snapshot
SEND	Creates a stream representation of the second snapshot
RECEIVE	Creates a snapshot whose contents are as specified in the stream provided
ALLOW	Displays permissions or Delegates ZFS administration permission for the file systems to non-privileged users
UNALLOW	Removes permissions that were granted with the zfs allow command
HOLD	Adds a single reference, named with the tag argument, to the specified snapshot or snapshots
HOLDS	Lists all existing user references for the given snapshot or snapshots
RELEASE	Removes a single reference, named with the tag argument, from the specified snapshot or snapshots
DIFF	Display the difference between a snapshot of a given filesystem and another snapshot of that filesystem from a later time or the current contents of the filesystem
PROPERTIES

CREATE

^

Creates a new ZFS file system

zfs create [-p] [-o property=value]... filesystem # Creates a new ZFS file system. The file system is automatically mounted according to the mountpoint property inherited from the parent.
  -o property=value # Sets the specified property as if the command zfs set property=value was invoked at the same time the dataset was created
  -p # Creates all the non-existing parent datasets. Datasets created in this manner are automatically mounted according to the mountpoint property inherited from their parent.

Creates a volume of the given size

zfs create [-ps] [-b blocksize] [-o property=value]... -V size volume # Creates a volume of the given size.  The volume is exported as a block device in /dev/zvol/path, where path is the name of the volume in the ZFS namespace
  -b blocksize # Equivalent to -o volblocksize=blocksize
  -o property=value # Sets the specified property as if the zfs set property=value command was invoked at the same time the dataset was created
  -p # Creates all the non-existing parent datasets
  -s # Creates a sparse volume with no reservation

examples

zfs create -o mountpoint=/var -o compression=lz4 $fs # create a filesytem with a mountpoint & compression options
zfs set quota=10G $fs # set a quota to user 'user'
zfs set compression=lz4 $fs# set lz4 compression for a fs
zfs set mountpoint=/var $fs # set mountpoint for a filesystem

DESTROY

^

Destroys the given dataset

zfs destroy [-Rfnprv] filesystem|volume
  -R # Recursively destroy all dependents, including cloned file systems outside the target hierarchy.
  -f # Force an unmount of any file systems using the unmount -f command
  -n # Do a dry-run ("No-op") deletion.  No data will be deleted
  -p # Print machine-parsable verbose information about the deleted data
  -r # Recursively destroy all children
  -v # Print verbose information about the deleted data

Destroys the given snapshot

The given snapshots are destroyed immediately if and only if the zfs destroy command without the -d option would have destroyed it

zfs destroy [-Rdnprv] filesystem|volume@snap[%snap[,snap[%snap]]]...
  -R # Recursively destroy all clones of these snapshots, including the clones, snapshots, and children. -d flag will have no effect
  -d # Defer snapshot deletion.
  -n # Do a dry-run ("No-op") deletion.  No data will be deleted
  -p # Print machine-parsable verbose information about the deleted data
  -r # Destroy (or mark for deferred deletion) all snapshots with this name in descendent file systems
  -v # Print verbose information about the deleted data

Destroys the given bookmark

zfs destroy filesystem|volume#bookmark

SNAPSHOT

^

Creates snapshots with the given names

zfs snapshot [-r] [-o property=value]... filesystem@snapname|volume@snapname...
  -o property=value # Sets the specified property; see zfs create for details
  -r # Recursively create snapshots of all descendent datasets

ROLLBACK

^

Roll back the given dataset to a previous snapshot

Roll back the given dataset to a previous snapshot. The -rR options do not recursively destroy the child snapshots of a recursive snapshot. Only direct snapshots of the specified filesystem are destroyed by either of these options. To completely roll back a recursive snapshot, you must rollback the individual child snapshots.

zfs rollback [-Rfr] snapshot
  -R # Destroy any more recent snapshots and bookmarks, as well as any clones of those snapshots
  -f # Used with the -R option to force an unmount of any clone file systems that are to be destroyed
  -r # Destroy any snapshots and bookmarks more recent than the one specified

CLONE

^

Creates a clone of the given snapshot

zfs clone [-p] [-o property=value]... snapshot filesystem|volume
  -o property=value # Sets the specified property
  -p # Creates all the non-existing parent datasets.  Datasets created in this manner are automatically mounted according to the mountpoint property inherited from their parent

PROMOTE

^

Promotes a clone file system to no longer be dependent on its "origin" snapshot

zfs promote clone-filesystem

RENAME

^

Renames dataset

Renames the given dataset. The new target can be located anywhere in the ZFS hierarchy, with the exception of snapshots. Snapshots can only be renamed within the parent file system or volume

zfs rename [-fp] filesystem|volume filesystem|volume
  -f # Force unmount any filesystems that need to be unmounted in the process
  -p # Creates all the nonexistent parent datasets.  Datasets created in this manner are automatically mounted according to the mountpoint property inherited from their parent

Renames snapshot

Recursively rename the snapshots of all descendent datasets. Snapshots are the only dataset that can be renamed recursively

zfs rename -r snapshot snapshot

LIST

^

Lists the property information for the given datasets in tabular form

zfs list [-r|-d depth] [-Hp] [-o property[,property]...] [-s property]... [-S property]... [-t type[,type]...] [filesystem|volume|snapshot]...
  -H # Used for scripting mode.  Do not print headers and separate fields by a single tab instead of arbitrary white space
  -S property # Same as the -s option, but sorts by property in descending order
  -d depth # Recursively display any children of the dataset, limiting the recursion to depth
  -o property # A comma-separated list of properties to display
  -p # Display numbers in parsable (exact) values
  -r # Recursively display any children of the dataset on the command line
  -s property # A property for sorting the output by column in ascending order based on the value of the property
  -t type # A comma-separated list of types to display, where type is one of filesystem, snapshot, volume, bookmark, or all

examples

zfs list -o name,used,available,readonly,exec,referenced,mountpoint,mounted,quota,clones
zfs list -t all -r $pool # print recursively space used by volumes

SET

^

Sets the property or list of properties to the given value(s) for each dataset. Only some properties can be edited

zfs set property=value [property=value]... filesystem|volume|snapshot...

GET

^

Displays properties for the given datasets

zfs get [-r|-d depth] [-Hp] [-o field[,field]...] [-s source[,source]...] [-t type[,type]...] all | property[,property]... filesystem|volume|snapshot|bookmark...
  name      Dataset name
  property  Property name
  value     Property value
  source    Property source.  Can either be local, default, temporary, inherited, or none (-)

  -H # Any headers are omitted, and fields are explicitly separated by a single tab instead of an arbitrary amount of space
  -d depth # Recursively display any children of the dataset, limiting the recursion to depth
  -o field # A comma-separated list of columns to display.  name,property,value,source is the default value
  -p # Display numbers in parsable (exact) values
  -r # Recursively display properties for any children
  -s source # A comma-separated list of sources to display
  -t type # A comma-separated list of types to display, where type is one of filesystem, snapshot, volume, bookmark, or all

INHERIT

^

Clears the specified property, causing it to be inherited from an ancestor, restored to default if no ancestor has the property set, or with the -S option reverted to the received value if one exists

zfs inherit [-rS] property filesystem|volume|snapshot...
  -r # Recursively inherit the given property for all children
  -S # Revert the property to the received value if one exists, otherwise operate as if the -S option was not specified

USERSPACE

^

Displays space consumed by, and quotas on, each user in the specified filesystem or snapshot. This corresponds to the userused@user, userobjused@user, userquota@user, and userobjquota@user properties.

zfs userspace [-Hinp] [-o field[,field]...] [-s field]... [-S field]... [-t type[,type]...] filesystem|snapshot
  -H # Do not print headers, use tab-delimited output
  -S field # Sort by this field in reverse order
  -i # Translate SID to POSIX ID.  The POSIX ID may be ephemeral if no mapping exists
  -n # Print numeric ID instead of user/group name
  -o field[,field]... # Display only the specified fields from the following set: type, name, used, quota.  The default is to display all fields.
  -p # Use exact (parsable) numeric output
  -s field # Sort output by this field.  The -s and -S flags may be specified multiple times to sort first by one field, then by another.  The default is -s type -s name
  -t # type[,type]... # Print only the specified types from the following set: all, posixuser, smbuser, posixgroup, smbgroup.  The default is -t posixuser,smbuser.  The default can be changed to include group types.

GROUPSPACE

^

Displays space consumed by, and quotas on, each group in the specified filesystem or snapshot

zfs groupspace [-Hinp] [-o field[,field]...] [-s field]... [-S field]... [-t type[,type]...] filesystem|snapshot

MOUNT

^

display

Displays all ZFS file systems currently mounted

zfs mount

mount

Mounts ZFS file systems

zfs mount [-Ov] [-o options] -a | filesystem
  -O # Perform an overlay mount
  -a # Mount all available ZFS file systems. Invoked automatically as part of the boot process
  filesystem # Mount the specified filesystem
  -o options # An optional, comma-separated list of mount options to use temporarily for the duration of the mount
  -v # Report mount progress

UNMOUNT

^

Unmounts currently mounted ZFS file systems

zfs unmount [-f] -a | filesystem|mountpoint
-a # Unmount all available ZFS file systems
filesystem|mountpoint # Unmount the specified filesystem
-f # Forcefully unmount the file system, even if it is currently in use

^

Shares available ZFS file systems

zfs share -a | filesystem
  -a # Share all available ZFS file systems
  filesystem # Share the specified filesystem according to the sharenfs and sharesmb properties

UNSHARE

^

Unshares currently shared ZFS file systems

zfs unshare -a | filesystem|mountpoint
-a # Unshare all available ZFS file systems
filesystem|mountpoint # Unshare the specified filesystem

BOOKMARK

^

Creates a bookmark of the given snapshot. Bookmarks mark the point in time when the snapshot was created, and can be used as the incremental source for a zfs send command

zfs bookmark snapshot bookmark

SEND

^

Creates a stream

Creates a stream representation of the second snapshot, which is written to standard output

zfs send [-DLPRcenpv] [[-I|-i] snapshot] snapshot
  -D, --dedup # Generate a deduplicated stream.  Blocks which would have been sent multiple times in the send stream will only be sent once.  The receiving system must also support this feature to receive a dedu‐
      plicated stream
  -I snapshot # Generate a stream package that sends all intermediary snapshots from the first snapshot to the second snapshot
  -L, --large-block # Generate a stream which may contain blocks larger than 128KB
  -P, --parsable # Print machine-parsable verbose information about the stream package generated
  -R, --replicate # Generate a replication stream package, which will replicate the specified file system, and all descendent file systems, up to the named snapshot
  -e, --embed # Generate a more compact stream by using WRITE_EMBEDDED records for blocks which are stored more compactly on disk by the embedded_data pool feature
  -c, --compressed # Generate a more compact stream by using compressed WRITE records for blocks which are compressed on disk and in memory
  -i # snapshot # Generate an incremental stream from the first snapshot (the incremental source) to the second snapshot (the incremental target)
  -n, --dryrun # Do a dry-run ("No-op") send
  -p, --props # Include the dataset's properties in the stream
  -v, --verbose # Print verbose information about the stream package generated

Generate a send stream

Generate a send stream, which may be of a filesystem, and may be incremental from a bookmark

zfs send [-Lce] [-i snapshot|bookmark] filesystem|volume|snapshot
  -L, --large-block # Generate a stream which may contain blocks larger than 128KB
  -c, --compressed # Generate a more compact stream by using compressed WRITE records for blocks which are compressed on disk and in memory
  -e, --embed # Generate a more compact stream by using WRITE_EMBEDDED records for blocks which are stored more compactly on disk by the embedded_data pool feature
  -i # snapshot|bookmark # Generate an incremental send stream

Generate a send stream

Creates a send stream which resumes an interrupted receive

zfs send [-Penv] -t receive_resume_token

RECEIVE

^

Creates a snapshot

Creates a snapshot whose contents are as specified in the stream provided on standard input

zfs receive [-Fnsuv] [-d|-e] [-o origin=snapshot] [-o property=value] [-x property] filesystem
  -F # Force a rollback of the file system to the most recent snapshot before performing the receive operation
  -d # Discard the first element of the sent snapshot's file system name
  -e # Discard all but the last element of the sent snapshot's file system name
  -n # Do not actually receive the stream
  -o origin=snapshot # Forces the stream to be received as a clone of the given snapshot
  -o property=value # Sets the specified property as if the command zfs set property=value was invoked immediately before the receive
  -s # If the receive is interrupted, save the partially received state, rather than deleting it
  -u # File system that is associated with the received stream is not mounted
  -v # Print verbose information about the stream and the time required to perform the receive operation
  -x property # Ensures that the effective value of the specified property after the receive is unaffected by the value of that property in the send stream (if any), as if the property had been excluded from the send stream

Abort an interrupted receive

Abort an interrupted zfs receive -s, deleting its saved partially received state

zfs receive -A filesystem|volume

ALLOW

^

Display

Displays permissions that have been delegated on the specified filesystem or volume

zfs allow filesystem|volume

Delegates permission

Delegates ZFS administration permission for the file systems to non-privileged users

zfs allow [-dglu] user|group[,user|group]... perm|@setname[,perm|@setname]... filesystem|volume zfs allow [-dl] -e|everyone perm|@setname[,perm|@setname]... filesystem|volume
  -d # Allow only for the descendent file systems
  -e|everyone # Specifies that the permissions be delegated to everyone
  -g group[,group]... # Explicitly specify that permissions are delegated to the group
  -l # Allow "locally" only for the specified file system
  -u user[,user]... # Explicitly specify that permissions are delegated to the user
  user|group[,user|group]... # Specifies to whom the permissions are delegated.  Multiple entities can be specified as a comma-separated list
  perm|@setname[,perm|@setname]... # The permissions to delegate.  Multiple permissions may be specified as a comma-separated list. Permissions are generally the ability to use a ZFS subcommand or change a ZFS property

Available permissions

NAME             TYPE           NOTES
allow            subcommand     Must also have the permission that isbeing allowed
clone            subcommand     Must also have the 'create' ability and 'mount' ability in the origin file system
create           subcommand     Must also have the 'mount' ability
destroy          subcommand     Must also have the 'mount' ability
diff             subcommand     Allows lookup of paths within a dataset given an object number, and the ability to create snapshots necessary to 'zfs diff'
mount            subcommand     Allows mount/umount of ZFS datasets
promote          subcommand     Must also have the 'mount' and 'promote' ability in the origin file system
receive          subcommand     Must also have the 'mount' and 'create' ability
rename           subcommand     Must also have the 'mount' and 'create' ability in the new parent
rollback         subcommand     Must also have the 'mount' ability
send             subcommand
share            subcommand     Allows sharing file systems over NFS or SMB protocols
snapshot         subcommand     Must also have the 'mount' ability

groupquota       other          Allows accessing any groupquota@... property
groupused        other          Allows reading any groupused@... property
userprop         other          Allows changing any user property
userquota        other          Allows accessing any userquota@... property
userused         other          Allows reading any userused@... property

aclinherit       property
acltype          property
atime            property
canmount         property
casesensitivity  property
checksum         property
compression      property
copies           property
devices          property
exec             property
filesystem_limit property
mountpoint       property
nbmand           property
normalization    property
primarycache     property
quota            property
readonly         property
recordsize       property
refquota         property
refreservation   property
reservation      property
secondarycache   property
setuid           property
sharenfs         property
sharesmb         property
snapdir          property
snapshot_limit   property
utf8only         property
version          property
volblocksize     property
volsize          property
vscan            property
xattr            property
zoned            property

time permission

Sets "create time" permissions

zfs allow -c perm|@setname[,perm|@setname]... filesystem|volume

to a set

Defines or adds permissions to a permission set

zfs allow -s @setname perm|@setname[,perm|@setname]... filesystem|volume

UNALLOW

^

Removes permissions

Removes permissions that were granted with the zfs allow command. No permissions are explicitly denied, so other permissions granted are still in effect

zfs unallow [-dglru] user|group[,user|group]... [perm|@setname[,perm|@setname]...] filesystem|volume zfs unallow [-dlr] -e|everyone [perm|@setname[,perm|@setname]...] filesystem|volume zfs unallow [-r] -c
  [perm|@setname[,perm|@setname]...] filesystem|volume
  -r # Recursively remove the permissions from this file system and all descendents

from a set

Removes permissions from a permission set

zfs unallow [-r] -s @setname [perm|@setname[,perm|@setname]...] filesystem|volume

HOLD

^

Adds a single reference, named with the tag argument, to the specified snapshot or snapshots

zfs hold [-r] tag snapshot...
  -r # Specifies that a hold with the given tag is applied recursively to the snapshots of all descendent file systems

HOLDS

^

Lists all existing user references for the given snapshot or snapshots

zfs holds [-r] snapshot...
  -r # Lists the holds that are set on the named descendent snapshots, in addition to listing the holds on the named snapshot

RELEASE

^

Removes a single reference, named with the tag argument, from the specified snapshot or snapshots

zfs release [-r] tag snapshot...
-r # Recursively releases a hold with the given tag on the snapshots of all descendent file systems

DIFF

^

Display the difference between a snapshot of a given filesystem and another snapshot of that filesystem from a later time or the current contents of the filesystem

zfs diff [-FHt] snapshot snapshot|filesystem
  -F # Display an indication of the type of file, in a manner similar to the - option of ls(1).
  -F B   Block device
  -F C   Character device
  -F /   Directory
  -F >   Door
  -F |   Named pipe
  -F @   Symbolic link
  -F P   Event port
  -F =   Socket
  -F F   Regular file

  -H # Give more parsable tab-separated output, without header lines and without arrows.

  -t # Display the path's inode change time as the first column of output.

The types of change are:

-   The path has been removed
+   The path has been created
M   The path has been modified
R   The path has been renamed

PROPERTIES

^

Properties are divided into two types, native and user properties. Native properties either export internal statistics or control ZFS behavior. In addition, native properties are either editable or read-only

available # The amount of space available to the dataset and all its children
compressratio # For non-snapshots, the compression ratio achieved for the used space of this dataset, expressed as a multiplier
createtxg # The transaction group (txg) in which the dataset was created
creation # The time this dataset was created
clones # For snapshots, this property is a comma-separated list of filesystems or volumes which are clones of this snapshot
defer_destroy # This property is on if the snapshot has been marked for deferred destroy by using the zfs destroy -d command
filesystem_count # The total number of filesystems and volumes that exist under this location in the dataset tree
guid # The 64 bit GUID of this dataset or bookmark which does not change over its entire lifetime
logicalreferenced # The amount of space that is "logically" accessible by this dataset
logicalused # The amount of space that is "logically" consumed by this dataset and all its descendents
mounted # For file systems, indicates whether the file system is currently mounted
origin # For cloned file systems or volumes, the snapshot from which the clone was created
receive_resume_token  For filesystems or volumes which have saved partially-completed state from zfs receive -s, this opaque token can be provided to zfs send -t to resume and complete the zfs receive
referenced # The amount of data that is accessible by this dataset, which may or may not be shared with other datasets in the pool
refcompressratio # The compression ratio achieved for the referenced space of this dataset, expressed as a multiplier
snapshot_count # The total number of snapshots that exist under this location in the dataset tree
type # The type of dataset: filesystem, volume, or snapshot
used # The amount of space consumed by this dataset and all its descendents
usedby* # The usedby* properties decompose the used properties into the various reasons that space is used
usedbychildren # The amount of space used by children of this dataset, which would be freed if all the dataset s children were destroyed
usedbydataset # The amount of space used by this dataset itself
usedbyrefreservation  The amount of space used by a refreservation set on this dataset
usedbysnapshots # The amount of space consumed by snapshots of this dataset
userused@user # The amount of space consumed by the specified user in this dataset
userobjused@user # The userobjused property is similar to userused but instead it counts the number of objects consumed by a user
userrefs # This property is set to the number of user holds on this snapshot
groupused@group # The amount of space consumed by the specified group in this dataset
groupobjused@group # The number of objects consumed by the specified group in this dataset
volblocksize # For volumes, specifies the block size of the volume
written # The amount of space referenced by this dataset
written@snapshot # The amount of referenced space written to this dataset since the specified snapshot

The following native properties can be used to change the behavior of a ZFS dataset

aclinherit=discard|noallow|restricted|passthrough|passthrough-x # Controls how ACEs are inherited when files and directories are created
acltype=off|noacl|posixacl # Controls whether ACLs are enabled and if so what type of ACL to use
atime=on|off # Controls whether the access time for files is updated when they are read
canmount=on|off|noauto # If this property is set to off, the file system cannot be mounted, and is ignored by zfs mount -a
checksum=on|off|fletcher2|fletcher4|sha256|noparity|sha512|skein|edonr # Controls the checksum used to verify data integrity
compression=on|off|gzip|gzip-N|lz4|lzjb|zle # Controls the compression algorithm used for this dataset
context=none|SELinux_User:SElinux_Role:Selinux_Type:Sensitivity_Level # This flag sets the SELinux context for all files in the file system under a mount point for that file system
fscontext=none|SELinux_User:SElinux_Role:Selinux_Type:Sensitivity_Level # This flag sets the SELinux context for the file system file system being mounted
defcontext=none|SELinux_User:SElinux_Role:Selinux_Type:Sensitivity_Level # This flag sets the SELinux default context for unlabeled files
rootcontext=none|SELinux_User:SElinux_Role:Selinux_Type:Sensitivity_Level # This flag sets the SELinux context for the root inode of the file system
copies=1|2|3 # Controls the number of copies of data stored for this dataset
devices=on|off # Controls whether device nodes can be opened on this file system
dnodesize=legacy|auto|1k|2k|4k|8k|16k # Specifies a compatibility mode or literal value for the size of dnodes in the file system.
exec=on|off # Controls whether processes can be executed from within this file system
filesystem_limit=count|none # Limits the number of filesystems and volumes that can exist under this point in the dataset tree
mountpoint=path|none|legacy # Controls the mount point used for this file system
nbmand=on|off # Controls whether the file system should be mounted with nbmand (Non Blocking mandatory locks)
overlay=off|on # Allow mounting on a busy directory or a directory which already contains files or directories
primarycache=all|none|metadata # Controls what is cached in the primary cache (ARC)
quota=size|none # Limits the amount of space a dataset and its descendents can consume
snapshot_limit=count|none # Limits the number of snapshots that can be created on a dataset and its descendents
userquota@user=size|none # Limits the amount of space consumed by the specified user
userobjquota@user=size|none # The userobjquota is similar to userquota but it limits the number of objects a user can create
groupquota@group=size|none # Limits the amount of space consumed by the specified group
groupobjquota@group=size|none # The groupobjquota is similar to groupquota but it limits number of objects a group can consume
readonly=on|off # Controls whether this dataset can be modified
recordsize=size # Specifies a suggested block size for files in the file system
redundant_metadata=all|most # Controls what types of metadata are stored redundantly
refquota=size|none # Limits the amount of space a dataset can consume
refreservation=size|none # The minimum amount of space guaranteed to a dataset, not including its descendents
relatime=on|off # Controls the manner in which the access time is updated when atime=on is set
reservation=size|none # The minimum amount of space guaranteed to a dataset and its descendants
secondarycache=all|none|metadata # Controls what is cached in the secondary cache (L2ARC)
setuid=on|off # Controls whether the setuid bit is respected for the file system
sharesmb=on|off|opts # Controls whether the file system is shared by using Samba USERSHARES and what options are to be used
sharenfs=on|off|opts # Controls whether the file system is shared via NFS, and what options are to be used
logbias=latency|throughput # Provide a hint to ZFS about handling of synchronous requests in this dataset
snapdev=hidden|visible # Controls whether the volume snapshot devices under /dev/zvol/<pool> are hidden or visible
snapdir=hidden|visible # Controls whether the .zfs directory is hidden or visible in the root of the file system as discussed in the Snapshots section
sync=standard|always|disabled # Controls the behavior of synchronous requests (e.g. fsync, O_DSYNC)
version=N|current # The on-disk version of this file system, which is independent of the pool version
volsize=size # For volumes, specifies the logical size of the volume
volmode=default|full|geom|dev|none # This property specifies how volumes should be exposed to the OS
vscan=on|off # Controls whether regular files should be scanned for viruses when a file is opened and closed
xattr=on|off|sa # Controls whether extended attributes are enabled for this file system
zoned=on|off # Controls whether the dataset is managed from a non-global zone

The following three properties cannot be changed after the file system is created, and therefore, should be set when the file system is created
If the properties are not set with the zfs create or zpool create commands, these properties are inherited from the parent dataset. If the parent dataset lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties.

casesensitivity=sensitive|insensitive|mixed # Indicates whether the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching
normalization=none|formC|formD|formKC|formKD # Indicates whether the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used
utf8only=on|off # Indicates whether the file system should reject file names that include characters that are not present in the UTF-8 character code set

bash · zfs

Sat Dec 14 14:33:17 2019 * · permalink

http://code.ambau.ovh/snippets/zfs.html

blackview - s8

MT6750T BIOS

auto test
manual test
single test
test report
debug test system
clear emmc
version information
restart

BIOS RESET

vol-Down + power # wait until robot appears

FACTORY RESET

vol-Up + power # wait until robot appears
power + vol-Up # keep power & press only one time on vol-up

SOFTWARE

DOWNLOAD

f-droid
whats-app
ampere

F-DROID

audio recorder
calendar notifications
osm and+
davx5
Dir
Document viewer
firefox klar
k-9 mail
libreoffice viewer
nextcloud
opencamera
openkeychain
pdf creator
simple flashlight
simple gallery pro
sms backup+
tasks
telegram
todo agenda for Android 4 - 7.0 
vanilla music
vanilla tag editor
vlc
wikipedia

MTP MOUNT POINT

/run/user/1000/gvfs/... #   manjaro
rsync /run/user/1000/gvfs/mtp\:host\=motorola_Moto_G__5__ZY3228VHQG/Carte\ SD/ Downloads/moto-g5/ -av --delete -n # rsync moto-g5

android

Mon Dec 9 10:18:51 2019 * · permalink

http://code.ambau.ovh/snippets/blackview_s8.html

rsync

A fast, versatile, remote (and local) file-copying tool

-v, --verbose      # increase verbosity 
-q, --quiet        # suppress non-error messages
-a, --archive      # archive mode; equals -rlptgoD (no -H,-A,-X)

-r, --recursive    # recurse into directories   
-l, --links        # copy symlinks as symlinks  
-p, --perms        # preserve permissions
-t, --times        # preserve modification times
-o, --owner        # preserve owner (super-user only)   
-g, --group        # preserve group
    --devices      # preserve device files (super-user only)    
    --specials     # preserve special files
-D                 # same as --devices --specials

-e # specify the remote ssh options to use

examples

rsync -rlptDv --delete -n # same as -av --delete but without preserving group & user
rsync -e "ssh -p 22" # use specified port 22 to connect to remote server

rsync Music/ root@node1:/save/vm/nextcloud/data/aguy/files/perso/music/ -rlptDv --delete -n

bash

Wed Dec 4 17:21:06 2019 * · permalink

http://code.ambau.ovh/snippets/bash_rsync.html

bash - syntax

https://ss64.com/bash/syntax.html

SYNTAX

command > filename # Redirect command output (stdout) into a file
command > /dev/null # Discard stdout of command
command 2> filename # Redirect error output (stderr) to a file
command 2>&1 filename # Redirect stderr to stdout 
command 1>&2 filename # Redirect stdout to stderr
command >> filename # Redirect command output and APPEND into a file
command < filename # Redirect a file into a command
command1 < (command2) # Redirect the output of command2 as file input to command1
command1 | tee filename | command2 # Redirect command1 into filename AND command2
command1 | command2 # Redirect stdout of command1 to command2
command1 |& command2 # Redirect stdERR of command1 to command2
command1 & command2 # Run command1 and then run command2 (asynchronous).
command1 ; command2 # Run command1 and afterwards run command2 (synchronous)
command1 && command2 # Run command2 only if command1 is successful (synchronous AND)
command1 || command2 # Run command2 only if command1 is NOT successful
command & # Run command in a subshell.
command &> filename # Redirect every output of command to filename
command > >(tee -a filename1 filename2) # Redirect command output (stdout) to stdout and into filename1 and filename2

# noclobber option can prevent overwriting an existing file

$ set -o noclobber turns ON noclobber
$ set +o noclobber turns OFF noclobber

[n]<word # Redirection of input causes the file whose name results from the expansion of word to be opened for reading on file descriptor n, or the standard input (file descriptor 0) if n is not specified.

[n]>[|]word # Redirection of output causes the file whose name results from the expansion of word to be opened for writing on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created; if it does exist it is truncated to zero size. If the redirection operator is '>', and the noclobber option to the set builtin has been enabled, the redirection will fail if the file whose name results from the expansion of word exists and is a regular file. If the redirection operator is '>|', or the redirection operator is '>' and the noclobber option is not enabled, the redirection is attempted even if the file named by word exists.

[n]>>word # Redirection of output in this fashion causes the file whose name results from the expansion of word to be opened for appending on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created.

# There are three formats for redirecting standard output and standard error:
&>word
>&word
>word 2>&1

ls > dirlist 2>&1 # directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command

ls 2>&1 > dirlist # directs only the standard output to file dirlist, because the standard error was duplicated as standard output before the standard output was redirected to dirlist.

DESCRIPTOR

exec 3< echolist # for reading
exec 3<&-
exec 3>&- # for writing
exec 3<&1.
exec 2> >(tee -a /tmp/2) > >(tee -a /tmp/1) 4>&1 # duplicate stderror & stdout in files & 4 in 1

examples

echo 1234567890 > $file # Write string to file
exec 3<> $file # Open $file and assign fd 3 to it
read -n 4 <&3 # Read only 4 characters
echo -n , >&3 # Write a decimal point there
exec 3>&- # Close fd 3
cat $file # show 1234.67890

SPECIAL FILE FOR REDIRECTIONS

/dev/fd/fd # If fd is a valid integer, file descriptor fd is duplicated
/dev/stdin # File descriptor 0 is duplicated
/dev/stdout # File descriptor 1 is duplicated
/dev/stderr # File descriptor 2 is duplicated
/dev/tcp/host/port # If host is a valid hostname or Internet address, and port is an integer port number, Bash attempts to open a TCP connection to the corresponding socket
/dev/udp/host/port # If host is a valid hostname or Internet address, and port is an integer port number, Bash attempts to open a UDP connection to the corresponding socket

HERE DOCUMENTS

This type of redirection instructs the shell to read input from the current source until a line containing only word (with no trailing blanks) is seen. All of the lines read up to that point are then used as the standard input for a command. If the redirection operator is '<<-', then all leading tab characters are stripped from input lines and the line containing delimiter. This allows here-documents within shell scripts to be indented in a natural fashion

<<[-]word
    here-document
word

HERE STRINGS

A here string can be considered as a stripped-down form of a here document.
It consists of nothing more than command <<<$word, where $word is expanded and fed to the stdin of command.

command <<<$word
command <<<"$word" # keep formatting

DUPLICATING FILE DESCRIPTORS

[n]<&word

Is used to duplicate input file descriptors. If word expands to one or more digits, the file descriptor denoted by n is made to be a copy of that file descriptor. If the digits in word do not specify a file descriptor open for input, a redirection error occurs. If word evaluates to '-', file descriptor n is closed. If n is not specified, the standard input (file descriptor 0) is used

[n]>&word

Is used similarly to duplicate output file descriptors. If n is not specified, the standard output (file descriptor 1) is used. If the digits in word do not specify a file descriptor open for output, a redirection error occurs. As a special case, if n is omitted, and word does not expand to one or more digits, the standard output and standard error are redirected as described previously

THE REDIRECTION OPERATOR

[n]<>word

causes the file whose name is the expansion of word to be opened for both reading and writing on file descriptor n, or on file descriptor 0 if n is not specified. If the file does not exist, it is created.

PROCESS SUBSTITUTION

>(commands) & <(commands)

The process list is run with its input or output connected to a FIFO or some file in /dev/fd. The name of this file is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Note that no space can appear between the < or > and the left parenthesis, otherwise the construct would be interpreted as a redirection.

examples

$(< file) is faster than $(cat file)

bash · bashroot

Fri Nov 29 08:27:46 2019 * · permalink

http://code.ambau.ovh/snippets/bash_syntax.html

bash - shortcuts

COMMAND EDITING

ctrl+ a # go to the start of the command line
ctrl+ e # go to the end of the command line
ctrl+ k # delete from cursor to the end of the command line
ctrl+ u # delete from cursor to the start of the command line
ctrl+ w # delete from cursor to start of word (i.e. delete backwards one word)
ctrl+ y # paste word or text that was cut using one of the deletion shortcuts (such as the one above) after the cursor
ctrl+ xx # move between start of command line and current cursor position (and back again)
alt+ b # move backward one word (or go to start of word the cursor is currently on)
alt+ f # move forward one word (or go to end of word the cursor is currently on)
alt+ d # delete to end of word starting at cursor (whole word if cursor is at the beginning of word)
alt+ c # capitalize to end of word starting at cursor (whole word if cursor is at the beginning of word)
alt+ u # make uppercase from cursor to end of word
alt+ l # make lowercase from cursor to end of word
alt+ t # swap current word with previous
ctrl+ f # move forward one character
ctrl+ b # move backward one character
ctrl+ d # delete character under the cursor
ctrl+ h # delete character before the cursor
ctrl+ t # swap character under cursor with the previous one

COMMAND RECALL

ctrl+ r # search the history backwards
ctrl+ g # escape from history searching mode
ctrl+ p # previous command in history (i.e. walk back through the command history)
ctrl+ n # next command in history (i.e. walk forward through the command history)
alt+ . # use the last word of the previous command

COMMAND CONTROL

ctrl+ l # clear the screen
ctrl+ s # stops the output to the screen (for long running verbose command)
ctrl+ q # allow output to the screen (if previously stopped using command above)
ctrl+ c # terminate the command
ctrl+ z # suspend/stop the command

BASH BANG COMMAND !

!! # run last command
!blah # run the most recent command that starts with ‘blah’ (e.g. !ls)
!blah:p # print out the command that !blah would run (also adds it as the latest command in the command history)
!$ # the last word of the previous command (same as alt+ .)
!$:p # print out the word that !$ would substitute
!* # the previous command except for the last word (e.g. if you type ‘_find somefile.txt /’, then !* would give you ‘_find somefile.txt’)
!*:p # print out what !* would substitute

bash · bashroot

Fri Nov 29 07:39:43 2019 * · permalink

http://code.ambau.ovh/snippets/bash_shortcut.html

tmux

https://tmuxcheatsheet.com/

COMMAND

'<>'' represents the shortcut to enter in command mode of tmux. Initialy ctrl+n, for me ctrl+q

<> : # in a tmux session launch a command line (abstract tmux command)

tmux ls
tmux list-sessions
tmux list-windows -t $session # list windows in session $session (or current)

tmux / tmux new / tmux new-session # launch a new session
tmux new -s $session # launch a new session with a name $session
<> : kill-session -t $session # kill session $session (name or number)
<> : kill-session -a # kill all sessions
<> : kill-session -a -t $session # kill all but session $session
<> : rename-session -t $session# rename session
<> : rename-window -t $session:$window # rename window

<> : detach / : attach -d # detach session
<> : a / : attach -t $session:$windows # attach session / windows (or current session)
<> : switch -t $session # switch to session $session

SHORTCUTS

session

<> + d # detach current session
<> + ) # move to next session
<> + ( # move to previous session
<> + s # interactive session selection

window

<> + c # create a new window in current session
<> + n # switch to next window
<> + p # switch to previous window
<> + $n # switch to a specific terminal $n (start from 0)
<> + , # rename current window
<> + w # interactive window selection

pane

<> + " # new horizontal pane below
<> + % # new vertical pane to the right
<> + x # kill pane
<> + z # toggle between pane zoom

<> + ctrl+n # switch to next pane
<> + ctrl+p # switch to previous pane
<> + { # swap the current pane with the previous pane
<> + } # swap the current pane with the next pane

<> + space # switch layout

<> + q # show panes number
<> + q $n # move to pane $n
<> + ! # convert pane to window

<> + left # move to left
<> + right # move to right
<> + up # move to up
<> + down # move to down

others

<> + ? # show all shortcuts
<> + t # show time in windows
<> + [ # enter in copy-mode (q or <esc> to quit)

perso

<> + R # Reload conf file
<> + m # enable mouse mode
<> + M # disable mouse mode
<> + k # kill window
<> + K # kill server

copy mode

ctrl + PageUp # enter in copy mode & scroll page up
alt + Up / alt+up # enter in copy mode & scroll page up

ctrl+s # search in history in copy mode

PLUGINS

https://github.com/tmux-plugins

RESURRECT

resurect / Persists tmux environment across system restarts
https://github.com/tmux-plugins/tmux-resurrect

<> + ctrl+s # to save current worspace (sessions)
<> + ctrl+r # to restore worspace (sessions)

The last saved worspace is a symlink 'last' to the active worspace in ~/.tmux/resurrect/ where are keeped history of saved works

autostart

sudo cat > /etc/systemd/system/tmux.service << EOF
[Unit]
Description=tmux default session (detached)
Documentation=man:tmux(1)

[Service]
Type=forking
User=nikita
Group=nikita
Environment=DISPLAY=:0
ExecStart=/usr/bin/tmux new-session -d

ExecStop=/home/${USER}/.tmux/plugins/resurect/scripts/save.sh
ExecStop=/usr/bin/tmux kill-server
KillMode=none

RestartSec=2

[Install]
WantedBy=default.target
EOF

systemctl enable tmux --now

bash · tmux

Fri Nov 22 17:47:12 2019 * · permalink

http://code.ambau.ovh/snippets/tmux.html

python - tuto

https://www.apc.univ-paris7.fr/Downloads/cosmobs/JCH/UniversiteOuverte/anaconda%20spyder.pdf

python

Sun Nov 10 10:57:33 2019 · permalink

http://code.ambau.ovh/snippets/python-tuto.html

sublime text

install

conf

sudo sh -c "echo '\n# sublime-text hack\n127.0.0.1\tsublimetext.com\n127.0.0.1\twww.sublimetext.com\n127.0.0.1\tlicense.sublimehq.com' >> /etc/hosts"
apt install -y iptables-persistent
ips="45.55.255.55"
for ip in ${ips}; do sudo iptables -A OUTPUT -d ${ip} -j DROP; done
path="/etc/iptables"
[ -d "${path}" ] || sudo mkdir "${path}"
sudo iptables-save -f /etc/iptables/rules.v4

build 3211

----- BEGIN LICENSE -----
Member J2TeaM
Single User License
EA7E-1011316
D7DA350E 1B8B0760 972F8B60 F3E64036
B9B4E234 F356F38F 0AD1E3B7 0E9C5FAD
FA0A2ABE 25F65BD8 D51458E5 3923CE80
87428428 79079A01 AA69F319 A1AF29A4
A684C2DC 0B1583D4 19CBD290 217618CD
5653E0A0 BACE3948 BB2EE45E 422D2C87
DD9AF44B 99C49590 D2DBDEE1 75860FD2
8C8BB2AD B2ECE5A4 EFC08AF2 25A9B864
------ END LICENSE ------

build 3176

—– BEGIN LICENSE —–
sgbteam
Single User License
EA7E-1153259
8891CBB9 F1513E4F 1A3405C1 A865D53F
115F202E 7B91AB2D 0D2A40ED 352B269B
76E84F0B CD69BFC7 59F2DFEF E267328F
215652A3 E88F9D8F 4C38E3BA 5B2DAAE4
969624E7 DC9CD4D5 717FB40C 1B9738CF
20B3C4F1 E917B5B3 87C38D9C ACCE7DD8
5F7EF854 86B9743C FADC04AA FB0DA5C0
F913BE58 42FEA319 F954EFDD AE881E0B
—— END LICENSE ——

packagecontrol

https://packagecontrol.io # packages info + shortcuts

COLOR SCHEME

package control / Install Package / PackageResourceViewer
package control / PackageResourceViewer / Open Resource / select scheme
package control / PackageResourceViewer / Theme - Default / Adaptive.sublime-theme
package control / PackageResourceViewer / Theme - Default / adaptive / Adaptive.sublime-theme

SIZE

/home/$USER/.config/sublime-text-3/Packages/Theme - Default/adaptive/Adaptive.sublime-theme
/home/$USER/.config/sublime-text-3/Packages/Theme - Default/Default.sublime-theme

PACKAGE

python

Anaconda
Anaconda turns your Sublime Text 3 in a full featured Python development IDE
Python Debugger
Graphical Debugger for Sublime Text
Python 3
Python 3 and Cython language bundles for Sublime Text and TextMate
DocBlockr_Python
Sublime Text DocBlockr for python. Simplifies writing docstring comments in Python

bash · python · soft

Sun Nov 10 09:49:37 2019 * · permalink

http://code.ambau.ovh/snippets/sublimetext.html

manjaro - pacman

https://wiki.manjaro.org/index.php?title=Pacman_Overview
https://wiki.archlinux.fr/pacman
https://community.chakralinux.org/t/how-to-use-pacman-to-search-for-install-upgrade-and-uninstall-packages

PACMAN CONF

all configurations for pacman are stored in:
/etc/pacman.conf

UPGRADE -U

yay -P --stats # show statistics
yay -Ps

UPGRADE -U

yay -U /var/cache/pacman/pkg/python-3.9.3-1-x86_64.pkg.tar.zst # install a specific version

IGNORE

/etc/pacman.conf

sudo sed -i '/^#IgnorePkg/a IgnorePkg    = libvirt' /etc/pacman.conf

sudo sed -i '/^#IgnorePkg/a Include = /etc/pacman.d/ignorepcks' /etc/pacman.conf
sudo sh -c "echo -e 'IgnorePkg    = libvirt\nIgnorePkg    = libvirt-python' > /etc/pacman.d/ignorepcks"

SYNC -S

pacman -Si $package # get informlation about package
pacman -Sl $repo # list all packages from repository

pacman -Sy # update local packages list
pacman -Syy # replace local packages list

pacman -Su # upgrade packages
pacman -Suu # upgrade & downgrade if needed

pacman -Syu #  update & upgrade
pacman -Syyu
pacman -Syyuu

pacman -Syuc # update, upgrade, clean

pacman -S package # install package
pacman -S core/package_name # install a specified repo/core package
pacman -S plasma-{desktop,mediacenter,nm} # install tree name packages
pacman -Ss $regex # search packages
pacman -Ssq $regex # search packages with minimal print
pacman -S $pack... # install packages
pacman -S $repo/$pack # install package from specific repo

REMOVE -R

pacman -R $pack... # remove packages
pacman -Rs $pack... # remove packages and dependencies
pacman -Ru $pack... # remove unneeded packages

pacman -Qdt # list orphane packages
pacman -Rsn $(pacman -Qdtq) # remove orphan packages

QUERY -Q

pacman -Qqettn # List of Native installed packages
pacman -Qqettm # List of AUR installed packages

DOWNGRADE

pamac install downgrade # install downgrade
downgrade $package # interactive downgrade package

PACMAN-MIRRORS

pacman-mirrors --country-list # get list of countries for repositories
pacman-mirrors -c France # select a country for repositories
pacman-mirrors -i # generate an interactive mirrolist
pacman-mirrors -c all # reset custom mirrorlist

PACTREE

pactree package_name # list all the packages recursively depending on an installed package
pactree -r package_name # whoneeds package_name

AUR REPO & YAOURT

https://wiki.manjaro.org/index.php/D%C3%A9p%C3%B4t_AUR_%28Arch_User_Repository%29

bash · linux · manjaro

Wed Nov 6 13:47:45 2019 * · permalink

http://code.ambau.ovh/snippets/manjaro-pacman.html

manjaro

POST-INSTALLATION

http://stephenmyall.com/manjaro/
https://dolys.fr/forums/topic/mon-installation-post-installation-manjaro/

AUDIO

For audio micro+headphone 08bb:2902

file=/lib/udev/rules.d/90-pulseaudio.rules
sudo cp -a ${file} ${file}.$(date +%s)
sudo sed -i '/08bb.*2902.*behringer-umc22/ s|^|#|' ${file}

LXD

cgroup v2 error messaging

file=/lib/udev/rules.d/90-pulseaudio.rules
part='systemd.unified_cgroup_hierarchy=0'
sudo cp -a ${file} ${file}.$(date +%s)
grep -q "^GRUB_CMDLINE_LINUX=.*${part}" ${file} || sudo sed -i '/^GRUB_CMDLINE_LINUX=/ s|"$| ${part}"|' ${file}
sudo update-grub

EFIVARFS

file=/etc/default/grub
part='efi=runtime'
sudo cp -a ${file} ${file}.$(date +%s)
grep -q "^GRUB_CMDLINE_LINUX=.*${part}" ${file} || sudo sed -i '/^GRUB_CMDLINE_LINUX=/ s|"$| ${part}"|' ${file}
sudo update-grub

&

file=/etc/mkinitcpio.conf
part='efivarfs'
sudo cp -a ${file} ${file}.$(date +%s)
grep -q "^MODULES=.*${part}" ${file} || sed -i '/^MODULES=/ s|)$| ${part})|' ${file}
sudo mkinitcpio -P

LOGIN-SCREEN

/etc/dconf/db/gdm.d/02-logo

[org/gnome/login-screen]
logo='/path/to/logo.png'

logo

gsettings set org.gnome.login-screen logo '/path/to/logo.png'
dconf # /org/gnome/login-screen/logo

https://wiki.archlinux.org/index.php/GDM

DEVICES

lsmod # show modules
lspci -k # show pci devices with used informations
inxi -Fxz # show details of device

VIDEO

inxi -G # show video informations
sudo mhwd -a pci nonfree 0300 # install non free driver
mhwd -li # show installed driver from mhwd

PRINT

yay -S system-config-printer manjaro-printer
sudo usermod -a -G cups $USER
sudo gpasswd -a $USER sys
sudo systemctl enable --now org.cups.cupsd.service

RADIOTRAY

sudo ln -sv /usr/lib/libjsoncpp.so.24 /usr/lib/libjsoncpp.so.22

KVM QEMU Virt-Manager

https://computingforgeeks.com/install-kvm-qemu-virt-manager-arch-manjar/

bash · linux · manjaro

Tue Nov 5 22:02:42 2019 * · permalink

http://code.ambau.ovh/snippets/manjaro.html

grub

GRUB COMMAND

press 'c' while launching to interrupt grub launching and entre in grub command

CHANGE GRUB DISPLAY RESOLUTION

vbeinfo # list available resolutions

modify in /etc/default/grub (ex: 800x600)

GRUB_GFXMODE=$resolution

CLEAR SECTOR 32

dd if=/dev/zero of=/dev/sda bs=512 count=1 seek=32
grub-install $device

USB3 MSI GAMING

add in /etc/default/grub

GRUB_CMDLINE_LINUX="iommu=soft"

LXD

/etc/default/grub

GRUB_CMDLINE_LINUX="... systemd.unified_cgroup_hierarchy=0

GRUB_TIMEOUT

Modify set timeout for choosed value in /boot/grub/grub.cfg

if [ "$recordfail_broken" = 1 ]; then
    cat << EOF
    if [ \$grub_platform = efi ]; then
        set timeout=${GRUB_RECORDFAIL_TIMEOUT:-30}
        if [ x\$feature_timeout_style = xy ] ; then
            set timeout_style=menu
        fi
    fi
    EOF
fi

MANJARO

OS real name for btrfs

file="/etc/grub.d/30_os-prober"
file_keep="$file.keep$(date +%s)"
if ! grep -q 'LONGNAME="${LONGNAME} ${BTRFSsubvol/#subvol=/}"' "$file"; then
    sudo cp -a "$file" "$file.keep$(date +%s)" && sudo chmod -x "$file_keep"
    sudo sed -i "/LONGNAME=\"\${LABEL}\"/ a\  else\n    LONGNAME=\"\${LONGNAME} \${BTRFSsubvol/#subvol=/}\"" "$file"
    sudo update-grub
fi

UBUNTU 18.04

Remove error in startup log : systemd-backlight@backlight:acpi_video0.service, add in /etc/default/grub

acpi_osi='!Windows 2012' 
acpi_backlight=vendor in GRUB_CMDLINE_LINUX_DEFAULT  in /etc/default/grub

BOOT ENTRY

https://linux.die.net/man/8/efibootmgr

efibootmgr

efibootmgr # list all boot entries
-v # list all boot entries with details
-B -b XXXX # delete boot entry number XXXX

example for 970g

efibootmgr -c -g -d /dev/sda1 -p 1 -w -L 'Manjaro' -l '\EFI\Manjaro\grubx64.efi'
efibootmgr -c -g -d /dev/sda1 -p 1 -w -L 'ubuntu' -l '\EFI\ubuntu\shimx64.efi'
efibootmgr -c -g -d /dev/sda1 -p 1 -w -L 'Windows Boot Manager' -l '\EFI\Microsoft/Boot\bootmgfw.efi'

>>

Timeout: 1 seconds
BootOrder: 0000,0001,0002
Boot0000* Manjaro HD(1,GPT,8e91a305-046d-4e90-8548-efca286325a7,0x800,0x32000)/File(\EFI\Manjaro\grubx64.efi)
Boot0001* ubuntu  HD(1,GPT,8e91a305-046d-4e90-8548-efca286325a7,0x800,0x32000)/File(\EFI\ubuntu\shimx64.efi)
Boot0002* Windows Boot Manager  HD(1,GPT,8e91a305-046d-4e90-8548-efca286325a7,0x800,0x32000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)

bash · boot · linux · manjaro · ubuntu

Thu Oct 31 21:00:25 2019 * · permalink

http://code.ambau.ovh/snippets/grub.html

mooc - python - 1 - sequence

TYPES

str
unicode
list
tuple
buffer
xrange

PRIORITY ORDER

x in s # True if an item of s is equal to x, else False
x not in s # False if an item of s is equal to x, else True
s + t # the concatenation of s and t
s * n , n * s # n shallow copies of s concatenated
s[i] # i'th item of s, origin 0
s[i:j] # slice of s from i to j
s[i:j:k] # slice of s from i to j with step k
len(s) # length of s  
min(s) # smallest item of s 
max(s) # largest item of s

METHOD

s.index(str) # return first position of str in s
s.count() # return then number of elements

SLICE

first included, second excluded (even with negative step)

s[start:end:step]

syntax

s[start:]
s[:end]

s[:] # duplicates element
s[start:-end] # from index1 to (index2 from last)
s[-start:-end] # from (index1 from last) to (index2 from last)
s[-start:end] # from (index1 from last) to index2 -> empty
s[-start:end:-step] # from (index1 from last) to (index2 from last)
s[::-1] # reverse
s[start:end:-2] # with start > end else return None # reverse a part with step
s[start;start] = [n] # insert element n in s at position start
s[start:end] = [] # unset elements between start and end (start included & end excluded)

examples

l = [0, 2, 4, 8, 16, 32, 64, 128]
l
l[-1:1:-2]
l[2:4] = [100, 200, 300, 400, 500]
l

mooc · python

Thu Oct 17 10:01:08 2019 * · permalink

http://code.ambau.ovh/snippets/mooc-python-2-sequence.html

ubuntu

256colors for nested tmux

20.04

export TERM=xterm-256color

turn off/disable bluetooth device startup

18.04

grep -n DEVICES_TO_DISABLE_ON_STARTUP /etc/default/tlp

auto login

sudo sh -c "echo '
# autologin
[Seat:*]
autologin-session=xubuntu
autologin-user=${USER}
autologin-user-timeout=0' >> /etc/lightdm/lightdm.conf

bash · linux · ubuntu

Sat Oct 5 19:57:03 2019 * · permalink

http://code.ambau.ovh/snippets/ubuntu.html

opendkim - postfix

INSTALL

install

apt-get install -y opendkim opendkim-tools

data

domain="17112018.fr"
path_keys="/etc/opendkim/keys"
dkim="dkim"

KEYS

mkdir -p ${path_keys}/${domain}
cd ${path_keys}/${domain}
opendkim-genkey --bits=2048 -s ${dkim} -d ${domain}
chown opendkim:opendkim ${dkim}.private
chmod g-rwx ${dkim}.private

test

opendkim-testkey -d ${domain} -s ${dkim} -k /etc/opendkim/keys/${domain}/${dkim}.private -vvv

CONF

/etc/opendkim.conf

AutoRestart             Yes
AutoRestartRate         10/1h
UMask                   002
Syslog                  yes
SyslogSuccess           Yes
LogWhy                  Yes

Canonicalization        relaxed/simple

ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable

Mode                    sv
PidFile                 /var/run/opendkim/opendkim.pid
SignatureAlgorithm      rsa-sha256

UserID                  opendkim:opendkim

Socket                  inet:12301@localhost

/etc/default/opendkim

SOCKET="inet:12301@localhost"

/etc/postfix/main.cf

milter_protocol = 2
milter_default_action = accept

# without spamassassin
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301
# with spamassassin
#smtpd_milters = unix:/spamass/spamass.sock, inet:localhost:12301
#non_smtpd_milters = unix:/spamass/spamass.sock, inet:localhost:12301

/etc/opendkim/TrustedHosts

127.0.0.1
localhost

# IP senders
$SENDER_IP

# Domains senders
*.${domain}

/etc/opendkim/KeyTable

${dkim}._domainkey.${domain} ${domain}:${dkim}:${path_keys}/${domain}/${dkim}.private

/etc/opendkim/SigningTable

*@${domain} ${dkim}._domainkey.${domain}

RESTART

systemctl restart postfix opendkim

bash · dkim · mail · opendkim · postfix

Sun Nov 25 09:01:42 2018 * · permalink

http://code.ambau.ovh/snippets/opendkim-postfix.html

postfix - relay

SENDER

/etc/postfix/main.cf

relayhost = [$receiver_ip]

RECEIVER

/etc/postfix/main.cf

myhostname = $domain_to_relay
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = $myhostname, localhost.localdomain, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 $sender_ip1 $sender_ip2
virtual_alias_maps = hash:/etc/postfix/virtual

alias

/etc/postfix/virtual

$email_alias $email_to_send

Compile modifications

postmap /etc/postfix/virtual
postfix reload
postmap -q $email_alias /etc/postfix/virtual

RESTART

systemctl restart postfix.service

bash · mail · postfix

Sat Nov 24 20:09:43 2018 * · permalink

http://code.ambau.ovh/snippets/postfix-relay.html

mail - exim4

ALIASES / EXIM4

http://debian-facile.org/doc:reseau:exim4:redirection-mails-locaux

# add alias to container
ls /vm/root/*/etc/aliases|xargs -L 1 sed -i "\$a root: tech@17112018.fr"
cat /vm/root/*/etc/aliases
vz-launch -y "newaliases" all
vz-launch -y "exim -bt root" all
vz-launch -y "systemctl restart exim4" all

# test alias
exim -bt root

bash · exim4 · mail

Sat Nov 24 19:42:23 2018 * · permalink

http://code.ambau.ovh/snippets/mail-exim4.html

mybb

INSTALL

get mybb-1.8.19.tgz & put it in $path_install

domain="17112018.fr"
subdomain="forum"
path_install="/var/share/www/17112018.fr"
path="$path_install/forum"
version="1.8.19"
file_mybb="mybb-${version}"

cd "$path_install"
tar xzf "${file_mybb}.tgz"
ln -sv "$file_mybb" "$subdomain"
ln -sv "$file_mybb/Upload/" "$subdomain"

PRECONF

cp -a "$subdomain/htaccess.txt" "$subdomain/.htaccess"
sed -i '/Order Deny,Allow/d' "$subdomain/.htaccess"
sed -i 's/Deny from all/Require all denied/' "$subdomain/.htaccess"

# add subdomain in apache conf
nano /etc/apache2/sites-available/$domain.conf
sysars
a2ensite ${domain}.conf

SETUP

https://${subdomain}.${domain} # launch

POSTCONF

systemctl restart apache2.service
nano inc/config.php

chmod 666 "$subdomain/inc/config.php" "$subdomain/inc/settings.php"
chmod 777 "$subdomain/cache/" "$subdomain/cache/themes/" "$subdomain/uploads/" "$subdomain/uploads/avatars/"

app · bash · mybb · php

Sat Nov 17 15:35:03 2018 * · permalink

http://code.ambau.ovh/snippets/mybb.html

mariadb - replication

! before creation of DBs !

MASTER

/etc/mysql/conf.d/master.cnf

[mysqld]
# master
server_id=1

log-bin
log-basename=master1
binlog_do_db="nextcloud"
binlog_do_db="postfixadmin"

# db
replicate_do_db="nextcloud"
replicate_do_db="postfixadmin"

GRANT REPLICATION SLAVE ON *.* TO 'replic'@'10.0.0.12%' IDENTIFIED BY '${pwd}'; # user

systemctl restart mariadb # restart

SLAVE

/etc/mysql/conf.d/slave.cnf

[mysqld]
# slave
server_id=2
relay-log=relay-bin
expire_logs_days=32 # master expire_logs_days setting + 2

# db
replicate_do_db="nextcloud"
replicate_do_db="postfixadmin"

mariadb create dump user

GRANT SELECT, RELOAD, SUPER, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'dump'@'localhost' IDENTIFIED BY '${pwd}';
#GRANT RELOAD, SUPER, REPLICATION CLIENT ON *.* TO 'dump'@'localhost' IDENTIFIED BY '${pwd}';
#GRANT SELECT, LOCK TABLES ON *.* TO 'dump'@'localhost';

mariadb create purge user

GRANT SUPER ON *.* TO 'purge'@'localhost' IDENTIFIED BY '${pwd}';

restart

systemctl restart mariadb

start

with mysql client

CHANGE MASTER TO
  MASTER_HOST='10.0.0.120',
  MASTER_USER='replic',
  MASTER_PASSWORD='${pwd}',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='master1-bin.000001',
  MASTER_LOG_POS=330,
  MASTER_CONNECT_RETRY=10;
START SLAVE;

CRON

look at 'cron'

/etc/cron.d/mariadb-dump-slave # dump from vm slave to host

# dump replicated mariadb databases & rotate files
cmd=/usr/local/bs/mysql-dump-slave
opts="${pwd}=${pwd} path2=$path"
0 1 * * * root test -x $cmd && $cmd $opts # every day at 01h00
# 0 */1 * * * root test -x $cmd && $cmd # every hour

Change rights

chmod 700 /etc/cron.d/mariadb-dump-slave

/etc/cron.d/ns388803-mariadb # get dump from an other host, put this file to secondary server to get file from the host server

# cron for get mariadb dump from ns388803 to local
host="ns398616"
*/10 * * * * root rsync -av root@node2:/save/mariadb/ /save/$host/mariadb/ >/dev/null 2&>1 # synchronize every 10mn

SYSTEM

MASTER LOGS

RESET MASTER;
//RESET MASTER TO 'master1-bin.000006';

PURGE MASTER LOGS TO 'master1-bin.000006';
PURGE MASTER LOGS BEFORE '2018-11-15 12:00:00';

FLUSH LOGS;

purge binary logs to file

PURGE BINARY LOGS TO 'master1-bin.000006';
PURGE BINARY LOGS BEFORE '2018-11-15 12:00:00';

SHOW

SHOW SLAVE STATUS\G;
SHOW MASTER STATUS;

DELAYED REPLICATION

STOP SLAVE;
CHANGE MASTER TO master_delay=3600;
START SLAVE;

DUMP ON MASTER

mysqldump -udump -p${pwd} ${db} --no-data --master-data --single-transaction > "${path2}/${file}_stru.sql"
mysqldump -udump -p${pwd} ${db} --no-create-info --master-data --single-transaction | gzip -c > "${path2}/${file}_data.sql.gz"

DUMP ON SLAVE

mysqldump -udump -p${pwd} ${db} --no-data --dump-slave --single-transaction | gzip -c > ${path2}/${file}_stru.sql.gz
mysqldump -udump -p${pwd} ${db} --no-create-info --dump-slave  --single-transaction| gzip -c > ${path2}/${file}_data.sql.gz

RESTORE FROM DUMP

SLAVE

# stop replication
user=; pwd=
mysql -u${user} -p${pwd} -e "STOP SLAVE;"
# show replication
mysql -u${user} -p${pwd} -e "SHOW SLAVE STATUS \G;"|head -n20

MASTER

user=; pwd=; db=; path2=; file=
# lock tables
mysql -u${user} -p${pwd} ${db} -e "FLUSH TABLES WITH READ LOCK;"
# dump
mysqldump -u${user} -p${pwd} ${db} --no-data --master-data --single-transaction | gzip -c > "${path2}/${file}_stru.sql.gz"
mysqldump -u${user} -p${pwd} ${db} --no-create-info --master-data --single-transaction | gzip -c > "${path2}/${file}_data.sql.gz"
# drop db
mysql -u${user} -p${pwd} -e "DROP DATABASE IF EXISTS ${db};"
# create db
mysql -u${user} -p${pwd} -e "CREATE DATABASE ${db};"
# show dbs
mysql -u${user} -p${pwd} -e "SHOW DATABASES;"
# restore dump
gzip -dc ${path2}/${file}_stru.sql.gz | mysql -u${user} -p${pwd} ${db} # erase if exists
gzip -dc ${path2}/${file}_data.sql.gz | mysql -u${user} -p${pwd} ${db}
# show status
mysql -u${user} -p${pwd} ${db} -e "SHOW MASTER STATUS;"
# unlock tables
mysql -u${user} -p${pwd} ${db} -e "UNLOCK TABLES;"

SLAVE

user=; pwd=; db=; path2=; file=
# drop db
mysql -u${user} -p${pwd} -e "DROP DATABASE IF EXISTS ${db};"
# create db
mysql -u${user} -p${pwd} -e "CREATE DATABASE ${db};"
# show dbs
mysql -u${user} -p${pwd} -e "SHOW DATABASES;"
# restore dump
gzip -dc ${path2}/${file}_stru.sql.gz | mysql -u${user} -p${pwd} ${db} # erase if exists
gzip -dc ${path2}/${file}_data.sql.gz | mysql -u${user} -p${pwd} ${db}
# show replication
mysql -u${user} -p${pwd} -e "SHOW SLAVE STATUS \G;"|head -n20
# change replication
mysql -u${user} -p${pwd} -e "CHANGE MASTER TO MASTER_LOG_FILE='';"
mysql -u${user} -p${pwd} -e "CHANGE MASTER TO MASTER_LOG_POS=;"
# start replication
mysql -u${user} -p${pwd} -e "START SLAVE;"
# show replication
mysql -u${user} -p${pwd} -e "SHOW SLAVE STATUS \G;"|head -n20

RESTORE FROM SCRATCH

MASTER: lock tables

mysql -u${user} -p${pwd} ${db} -e "FLUSH TABLES WITH READ LOCK;"
mysql -u${user} -p${pwd} ${db} -e "SHOW MASTER STATUS;"

MASTER: dump master container with keeping MASTER position

# all databases
mysqldump -u${user} -p${pwd} --all-databases --master-data --single-transaction | gzip -c > ${path2}/all.sql.gz
#
# or list of databases in ${dbs}
mysqldump -u${user} -p${pwd} ${db} -e "FLUSH TABLES WITH READ LOCK;"
for db in ${dbs}; do
mysqldump -u${user} -p${pwd} ${db} --no-data --master-data --single-transaction | gzip -c > ${path2}/${file}_stru.sql.gz
mysqldump -u${user} -p${pwd} ${db} --no-create-info --master-data --single-transaction | gzip -c > ${path2}/${file}_data.sql.gz
done

MASTER: unlock tables

mysql -u${user} -p${pwd} ${db} -e "SHOW MASTER STATUS;"
mysql -u${user} -p${pwd} ${db} -e "UNLOCK TABLES;"

SLAVE: stop & reset slave service

mysql -u${user} -p${pwd} -e "STOP SLAVE; RESET SLAVE;"

load master data to slave

# all databases
gzip -dc ${path2}/all.sql.gz | mysql -u${user} -p${pwd}
#
# or list of databases in ${dbs}
for db in ${dbs}; do
gzip -dc ${path2}/${file}_stru.sql.gz | mysql -u${user} -p${pwd} ${db} # erase if exists
gzip -dc ${path2}/${file}_data.sql | mysql -u${user} -p${pwd} ${db}
done

start mariadb service in slave

mysql -u${user} -p${pwd} -e "START SLAVE;"

show state

mysql -u${user} -p${pwd} -e "START SLAVE;"
mysql -u${user} -p${pwd} -e "SHOW SLAVE STATUS\G;"

mariadb · mysql

Sat Nov 17 09:57:16 2018 * · permalink

http://code.ambau.ovh/snippets/mariadb-replication.html

mooc - python - 6 - context manager

https://docs.python.org/3/reference/datamodel.html#with-statement-context-managers

Allow to execute codes define in object himself in enter & exit methods
Propagates an exception based on the return value of the exit method:

False -> propagate
True -> does not propagate

Difference between try/finally & with/as are about the place of codes executed in finally or enter/exit:

for try/finally, codes are in codes arround object
for with/as, codes are in embeded in object

syntax

with expression as name:
    ...

catch specific exception in __exit__
    def __exit__(self, exc_type, exc_value, traceback):

example

with as

import time
class Timer:
    def __enter__(self):
        print("Entering Timer")
        self.start = time.time()
        return self

    def __exit__(self, *args):
        print(f"execution time {time.time() - self.start}")
        # propagate exception
        return False

    def __str__(self):
        return f"delay {time.time() - self.start}s"

with Timer() as t:
    sum(x for x in range(10_000_000))
    1/0
    print(t)
    sum(x**2 for x in range(10_000_000))

or

import time
try:
    with Timer():
        time.sleep(0.5)
        1/0
except Exception as exc:
    # on va bien recevoir cette exception
    print(f"OOPS -> {type(exc)}")

try finally

import time
try:
    start = time.time()
    print(sum(x for x in range(10_000_000)))
    1/0
    print(f"delay {time.time() - start}s")
    print(sum(x**2 for x in range(5_000_000)))
except ZeroDivisionError as e:
    print(f"OOPS, {type(e)}, {e}")
finally:
    print(f"execution time {time.time() - start}")

catch specific exception

class Timer2:
    def __enter__(self):
        print("Entering Timer1")
        self.start = time.time()
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        if exc_type is None:
            # pas d'exception levée dans le corps du 'with'
            print(f"Total duration {time.time()-self.start:2f}")
            # dans ce cas la valeur de retour n'est pas utilisée
        else:
            if exc_type in (ZeroDivisionError,) :
                print("on étouffe")
                # on peut l'étouffer en retournant True
                return True
            else:
                print(f"OOPS : on propage l'exception "
                      f"{exc_type} - {exc_value}")
                # et pour ça il suffit... de ne rien faire du tout
                # ce qui renverra None

with a filtered exception

try:
    with Timer2():
        time.sleep(0.5)
        1/0
except Exception as e:
    # on va bien recevoir cette exception
    print(f"OOPS -> {type(e)}")

with an other exception

try:
    with Timer2():
        time.sleep(0.5)
        raise OSError()
except Exception as e:
    # on va bien recevoir cette exception
    print(f"OOPS -> {type(e)}")

decorator

example

from contextlib import contextmanager

@contextmanager
def compact_timer(message):
    start = time.time()
    yield
    print(f"{message}: duration = {time.time() - start}")

with compact_timer("Squares sum"):
    print(sum(x**2 for x in range(10**5)))

mooc · python

Sat Nov 3 20:31:24 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-6-context-manager.html

mooc - python - 6 - class: special methods

https://docs.python.org/3/reference/datamodel.html#specialnames
https://docs.python.org/3/reference/datamodel.html#object.__new__
https://docs.python.org/3/reference/datamodel.html#object.__repr__

__init__ # called after the constructor and implement instance of class
__repr__ # called when the object calls without printing. if missing, use __str__ instead
__str__ # print when print method is called on object 
__bool__ # used for test on object
__len__ # called in place of __bool__ if missing
__add__, __mul__, __sub__, __div__, __and__, ... called by sum, +, *, ... methods
__contains__ # called by 'in' method
__len__ # called by 'len' method
__getitem__ # used by slice, calling element[], make element an iterable
__eq__ # called to compare object '==', 'is'. depends of hash, if __eq__ are implemented __hash__ have to be it also
__hash__ # https://docs.python.org/3/reference/datamodel.html?highlight=__hash__#object.__hash__
__call__ # use instance (object) of class like a function

EXAMPLES

call

class PlusClosure:
    def __init__(self, initial):
        self.initial = initial
    def __call__(self, *args):
        return self.initial + sum(args)

plus2 = PlusClosure(2)
plus2(5)

getattr

class RPCProxy:

    def __init__(self, url, login, password):
        self.url = url
        self.login = login
        self.password = password

    def __getattr__(self, function):
        """
        Crée à la volée une méthode sur RPCProxy qui correspond
        à la fonction distante 'function'
        """
        def forwarder(*args):
            print(f"Envoi à {self.url}...")
            print(f"de la fonction {function} -- args= {args}")
            return "retour de la fonction " + function
        return forwarder

rpc_proxy = RPCProxy (url='http://cloud.provider.com/JSONAPI',
                      login='dupont',
                      password='***')

# cette partie du code, en tant qu'utilisateur de l'API, est supposée connaître les détails des arguments à passer et de comment utiliser les valeurs de retour
nodes_list = rpc_proxy.GetNodes (
    [ ('phy_mem', '>=', '32G') ] )

# réserver un noeud
node_lease = rpc_proxy.BookNode (
    { 'id' : 1002, 'phy_mem' : '32G' } )

Point

class Point:
    a = 0
    b = 0
    d = [a, b]
    def __init__(self, x=0, y=0, len=0):
        self.x = x
        self.y = y
        self.p = [x, y]
    def __repr__(self):
        return f"Pt[{self.x}, {self.y}]"

class Point_eh(Point):
    def __eq__(self, other):
        return self.x == other.x and self.y == other.y
    def __hash__(self):
        return (11 * self.x + self.y)

p1 = Point(0,1)
p2 = Point(0,1)
p3 = Point(0,1)
s = {p1, p2}
p1
s

# __eq__
p1 == p2
peh1 = Point_eh(0,1)
peh2 = Point_eh(0,1)
s = {peh1, peh2}

Matrix

class Matrix:

    def __init__(self, *args):
        """
        le constructeur accepte
        (*) soit les 4 coefficients individuellement
        (*) soit une liste - ou + généralement une séquence - des mêmes
        """
        # on veut pouvoir créer l'objet à partir des 4 coefficients
        # souvenez-vous qu'avec la forme *args, args est toujours un tuple
        if len(args) == 4:
            self.coefs = args
        # ou bien d'une séquence de 4 coefficients
        elif len(args) == 1:
            self.coefs = tuple(*args)

    def __repr__(self):
        "l'affichage"
        return "[" + ", ".join([str(c) for c in self.coefs]) + "]"

    def __add__(self, other):
        """
        l'addition de deux matrices retourne un nouvel objet
        la possibilité de créer une matrice à partir
        d'une liste rend ce code beaucoup plus facile à écrire
        """
        return Matrix([a + b for a, b in zip(self.coefs, other.coefs)])

    def __bool__(self):
        """
        on considère que la matrice est non nulle
        si un au moins de ses coefficients est non nul
        """
        # ATTENTION le retour doit être un booléen
        # ou à la rigueur 0 ou 1
        for c in self.coefs:
            if c:
                return True
        return False

Remarquez que les opérandes sont apparemment inversés dans le sens où pour evaluer 'reel * matrice'

On écrit une méthode qui prend en argument la matrice, puis le réel mais n'oubliez pas qu'on est en fait en train d'écrire une méthode sur la classe Matrix2

def multiplication_scalaire(self, alpha):
    return Matrix2([alpha * coef for coef in self.coefs])

on ajoute la méthode spéciale rmul

Matrix2.__rmul__ = multiplication_scalaire

mooc · python

Sat Nov 3 20:30:58 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-6-class-special-methods.html

mooc - python - 6 - class: heritage

syntax

class Myclass(parentclass1, parentClass2, ...):

implicit
If the girl class does not define the method at all;
redefined
If one rewrites the method entirely;
modified
if we rewrite the method in the class girl, but using the code of the mother class.

super().method_name() can be used in place of Class_parent_name.method_name(self)

example

class Fleur:
    def implicite(self):
        print('Fleur.implicite')
    def redefinie(self):
        print('Fleur.redéfinie')
    def modifiee(self):
        print('Fleur.modifiée')

class Rose(Fleur):
    def redefinie(self):
        print('Rose.redefinie')
    def modifiee(self):
        Fleur.modifiee(self)
        print('Rose.modifiee apres Fleur')

# with use of super()
class Rose(Fleur):
    def modifiee(self):
        super().modifiee()
        print('Rose.modifiee apres Fleur')

COMPOSITION

class Tige:
    def implicite(self):
        print('Tige.implicite')
    def redefinie(self):
        print('Tige.redefinie')
    def modifiee(self):
        print('Tige.modifiee')

class Rose:
    def __init__(self):
        self.externe = Tige()
    def implicite(self):
        self.externe.implicite()
    def redefinie(self):
        print('Rose.redefinie')
    def modifiee(self):
        self.externe.modifiee()
        print('Rose.modifiee apres Tige')

NAMEDTUPLE

The objects will be unmutable (in fact they are tuples)
Incidentally we can access the different fields by their name as well as by an index

example

from collections import namedtuple
TuplePoint = namedtuple('TuplePoint', ['x', 'y'])
p3 = TuplePoint(1, 2)
p3.x
p3[0]

class Point2(namedtuple('Point2', ['x', 'y'])):

    # l'égalité va se baser naturellement sur x et y
    def __eq__(self, other):
        return self.x == other.x and self.y == other.y

    # du coup la fonction de hachage
    # dépend aussi de x et de y
    def __hash__(self):
        return (11 * self.x + self.y) // 16

q1 is q2
q1 == q2
s = {q1, q2}
q3 in s
# unmutable
try:
    q1.x = 100
except Exception as e:
    print(f"OOPS {type(e)}")

DATACLASS

https://docs.python.org/3/library/dataclasses.html

Record data class
frozen=True -> unmutable

example

from dataclasses import dataclass
>>>
@dataclass(frozen=True)
class Point:
    x: float
    y: float
    def __eq__(self, other):
        return self.x == other.x and self.y == other.y
    def __hash__(self):
        return (11 * self.x + self.y) // 16

p1, p2, p3 = Point(1, 1), Point(1, 1), Point(1, 1)
s = {p1, p2}
len(s)
p3 in s
try:
    p1.x = 10
except Exception as e:
    print(f"OOPS {type(e)}")

MULTIPLE HERITAGE

class Myclass(First_parent_class, Second_parent_class)

Order of attribute resolution is order of declaration of parent class

class C:
c = C()

Return the class name of object

c.__class__
C.__class__

Return direct parent (super) classes of object (not instance)

c.__bases__

Return order to interprete attribute resolution

C.__mro__
C.mro()

example

# STRANGE ORDER
class A1:
    pass
class B1:
    pass
class C1:
    pass
class A2(A1, C1):
    pass
class B2(B1):
    pass
class C3(A2, B2, C1):
    pass
C3.mro()

SCOPE

For class attributes, the resolution order follow the tree of heritage

example

a = 1
class C():
    a = 2
    class B():
        def f(self):
            print(a)
            print(C.a)
c = C.B()
c.f()

SCOPE

For class attributes, the resolution order follow the tree of heritage

example

class A():
    def __init__(self):
        print('A')

class B(A):
    def __init__(self):
        #super(B, self).__init__()
        print('B')

class C(B):
    def __init__(self):
        super().__init__()
        print('C')

class D(C):
    def __init__(self):
        super(B, self).__init__()
        print('D')

a = A()
print('')
b = B()
print('')
c = C()
print('')
d = D()
print('')

mooc · python

Sat Nov 3 20:30:03 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-6-class-heritage.html

mooc - python - 6 - class

GLOBAL

https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers
https://docs.python.org/3/tutorial/classes.html#tut-private

A Class & class instance is mutable
Inheritance tree: instance -> class -> parent class....
Attributes are calculate dynamicaly in time
Instance & class are shared same attributes
Self for instance is just a convention

class Test:
    att = 0
t = Test
s = "Here is !"
class Myclass:
    def initia(self, s):
    self.word = s
p = Myclass()

p.initia(s)

Myclass.initia(p,s)

VARS

vars(Myclass) <=> Myclass.__dict__ # return space name

DYNAMIC OVERLOAD

class Test:
     att = 0

def addatt(self, val):
    self.att = self.att + val

Test.addatt = addatt

PROPERTY

https://docs.python.org/3.6/library/functions.html#property

To use an encapsulation with attributes, use property to define the setters or getters, the calling attributes launch automatically the setters or getters

ATTRIBUTES CLASS VS INSTANCE

Class attribute defined in class are shared with all instances
Attributes reference are shared & changed according to whether it is mutable or not

class Point:
    a = 0
    b = 0
    d = [a, b]
    def __init__(self, x=0, y=0, len=0):
        self.x = x
        self.y = y
        self.p = [x, y]
    def __repr__(self):
        return f"Pt[{self.x}, {self.y}]"

# class attribute
t = Point()
print(f"t.d {t.d}", f"t.p {t.p}", f"Point.d {Point.d}", sep="\n")
print(f"Point.p {Point.p}")

# class -> instance
Point.a = 1
t2 = Point()
print(f"Point.a {Point.a}", f"t.a {t.a}", f"t2.a {t2.a}", sep="\n")

# attribute not mutable: instance !-> class
# loose reference
t2.a = 2
print(f"Point.a {Point.a}", f"t.a {t.a}", f"t2.a {t2.a}", sep="\n")
Point.a = 3
print(f"Point.a {Point.a}", f"t.a {t.a}", f"t2.a {t2.a}", sep="\n")

# attribute mutable: instance -> class
t2.d.append(3)
print(f"Point.d {Point.d}", f"t.d {t.d}", f"t2.d {t2.d}", sep="\n")
Point.d.append(4)
print(f"Point.d {Point.d}", f"t.d {t.d}", f"t2.d {t2.d}", sep="\n")

## instance attribute
t = Point()
print(f"t.p {t.p}", f"t2.p {t2.p}", sep="\n")
t.p.append(1)
print(f"t.p {t.p}", f"t2.p {t2.p}", sep="\n")

INSTANCE: ID, NOT VALUES

2 instances are compared by the id not values

p1 = Point1(2, 3)
p2 = Point1(2, 3)
p3 = Point1(2, 3)

Equality test instance id, not class values

p1 == p2

In test instance id, not class values

s = {p1, p2}
p1 in s
p3 in s

See examples in class-special-methods for hash eq__

GETATTR, SETATTR & HASATTR

import math
math.pi
hasattr(math, 'pi')
getattr(math, 'pi')
getattr(math, 'pi2', 'not found')
setattr(math, 'pi', math.pi*2)
math.pi

ENUM

https://docs.python.org/3/library/enum.html

enumeration

from enum import Enum
class Flavour(Enum):
    CHOCOLATE = 1
    VANILLA = 2
    PEAR = 3

chocolate = Flavour['CHOCOLATE']
chocolate
chocolate.name

INTENUM

https://docs.python.org/3/library/enum.html

IntEnum is Enum with comparison

from enum import IntEnum
class HttpError(IntEnum):
    OK = 200
    REDIRECT = 301
    REDIRECT_TMP = 302
    NOT_FOUND = 404
    INTERNAL_ERROR = 500

    def is_redirect(self):
        return 300

example

class Temperature:
    K = 273.16
    RF = 5 / 9
    KF = (K / RF) - 32

    def __init__(self, kelvin=None, celsius=None, fahrenheit=None):
        if kelvin is not None:
            self.kelvin = kelvin
        elif celsius is not None:
            self.celsius = celsius
        elif fahrenheit is not None:
            self.fahrenheit = fahrenheit
        else:
            self.kelvin = 0
            raise ValueError("need to specify at least one unit")

    def __repr__(self):
        return f""

    def __str__(self):
        return f"{self.kelvin:g}K"

    def _get_kelvin(self):
        return self._kelvin

    def _set_kelvin(self, kelvin):
        if kelvin < 0:
            raise ValueError(f"Kelvin {kelvin} must be positive")
        self._kelvin = kelvin

    kelvin = property(_get_kelvin, _set_kelvin)

    # les deux autres properties font la conversion, puis
    # sous-traitent à la property kelvin pour le contrôle de borne
    def _set_celsius(self, celsius):
        # using .kelvin instead of ._kelvin to enforce
        self.kelvin = celsius + self.K

    def _get_celsius(self):
        return self._kelvin - self.K

    celsius = property(_get_celsius, _set_celsius)

    def _set_fahrenheit(self, fahrenheit):
        # using .kelvin instead of ._kelvin to enforce
        self.kelvin = (fahrenheit + self.KF) * self.RF

    def _get_fahrenheit(self):
        return self._kelvin / self.RF - self.KF

    fahrenheit = property(_get_fahrenheit, _set_fahrenheit)
t = Temperature(200)
t
t.kelvin
t.kelvin = -30
t

complete example

class Temperature:

    ## les constantes de conversion
    # kelvin / celsius
    K = 273.16
    # fahrenheit / celsius
    RF = 5 / 9
    KF = (K / RF) - 32

    def __init__(self, kelvin=None, celsius=None, fahrenheit=None):
        """
        Création à partir de n'importe quelle unité
        Il faut préciser exactement une des trois unités
        """
        # on passe par les properties pour initialiser
        if kelvin is not None:
            self.kelvin = kelvin
        elif celsius is not None:
            self.celsius = celsius
        elif fahrenheit is not None:
            self.fahrenheit = fahrenheit
        else:
            self.kelvin = 0
            raise ValueError("need to specify at least one unit")

    # pour le confort
    def __repr__(self):
        return f""

    def __str__(self):
        return f"{self.kelvin:g}K"

    # l'attribut 'kelvin' n'a pas de conversion à faire,
    # mais il vérifie que la valeur est positive
    def _get_kelvin(self):
        return self._kelvin

    def _set_kelvin(self, kelvin):
        if kelvin

mooc · python

Sat Nov 3 20:24:58 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-6-class.html

letsencrypt - haproxy

DOMAINS

domains_17112018_fr="17112018.fr,ada.17112018.fr,admin.17112018.fr,blog.17112018.fr,carte.17112018.fr,chat.17112018.fr,chiffres.17112018.fr,cloud.17112018.fr,cms.17112018.fr,code.17112018.fr,compta.17112018.fr,datadoghq.17112018.fr,democratie.17112018.fr,dev.17112018.fr,diaspora.17112018.fr,discord.17112018.fr,discourse.17112018.fr,down.17112018.fr,elastik.17112018.fr,fil.17112018.fr,filerun.17112018.fr,files.17112018.fr,forum.17112018.fr,gestion.17112018.fr,git.17112018.fr,gitea.17112018.fr,gitlab.17112018.fr,goaccess.17112018.fr,grav.17112018.fr,graylog.17112018.fr,info.17112018.fr,kibana.17112018.fr,lcr.17112018.fr,liens.17112018.fr,links.17112018.fr,log.17112018.fr,manage.17112018.fr,matomo.17112018.fr,metrics.17112018.fr,monitor.17112018.fr,nextcloud.17112018.fr,pfa.17112018.fr,piwik.17112018.fr,plumxml.17112018.fr,pma.17112018.fr,roundcube.17112018.fr,shaarli.17112018.fr,snippet.17112018.fr,social.17112018.fr,st.17112018.fr,statistiques.17112018.fr,stats.17112018.fr,test.17112018.fr,tiddly.17112018.fr,tuleap.17112018.fr,vma.17112018.fr,vmail.17112018.fr,webmail.17112018.fr,wiki.17112018.fr,www.17112018.fr,zabbix.17112018.fr"

domains_17112018_ovh="17112018.ovh,ada.17112018.ovh,admin.17112018.ovh,blog.17112018.ovh,carte.17112018.ovh,chat.17112018.ovh,chiffres.17112018.ovh,cloud.17112018.ovh,cms.17112018.ovh,code.17112018.ovh,compta.17112018.ovh,datadoghq.17112018.ovh,democratie.17112018.ovh,dev.17112018.ovh,diaspora.17112018.ovh,discord.17112018.ovh,discourse.17112018.ovh,down.17112018.ovh,elastik.17112018.ovh,fil.17112018.ovh,filerun.17112018.ovh,files.17112018.ovh,forum.17112018.ovh,gestion.17112018.ovh,git.17112018.ovh,gitea.17112018.ovh,gitlab.17112018.ovh,goaccess.17112018.ovh,grav.17112018.ovh,graylog.17112018.ovh,info.17112018.ovh,kibana.17112018.ovh,lcr.17112018.ovh,liens.17112018.ovh,links.17112018.ovh,log.17112018.ovh,manage.17112018.ovh,matomo.17112018.ovh,metrics.17112018.ovh,monitor.17112018.ovh,nextcloud.17112018.ovh,pfa.17112018.ovh,piwik.17112018.ovh,plumxml.17112018.ovh,pma.17112018.ovh,roundcube.17112018.ovh,shaarli.17112018.ovh,snippet.17112018.ovh,social.17112018.ovh,st.17112018.ovh,statistiques.17112018.ovh,stats.17112018.ovh,test.17112018.ovh,tiddly.17112018.ovh,tuleap.17112018.ovh,vma.17112018.ovh,vmail.17112018.ovh,webmail.17112018.ovh,wiki.17112018.ovh,www.17112018.ovh,zabbix.17112018.ovh"

domains_ambau_ovh="ambau.ovh,ada.ambau.ovh,admin.ambau.ovh,blog.ambau.ovh,carte.ambau.ovh,chat.ambau.ovh,chiffres.ambau.ovh,cloud.ambau.ovh,cms.ambau.ovh,code.ambau.ovh,compta.ambau.ovh,datadoghq.ambau.ovh,democratie.ambau.ovh,dev.ambau.ovh,diaspora.ambau.ovh,discord.ambau.ovh,discourse.ambau.ovh,down.ambau.ovh,elastik.ambau.ovh,fil.ambau.ovh,filerun.ambau.ovh,files.ambau.ovh,forum.ambau.ovh,gestion.ambau.ovh,git.ambau.ovh,gitea.ambau.ovh,gitlab.ambau.ovh,goaccess.ambau.ovh,grav.ambau.ovh,graylog.ambau.ovh,info.ambau.ovh,kibana.ambau.ovh,lcr.ambau.ovh,liens.ambau.ovh,links.ambau.ovh,log.ambau.ovh,manage.ambau.ovh,matomo.ambau.ovh,metrics.ambau.ovh,monitor.ambau.ovh,nextcloud.ambau.ovh,pfa.ambau.ovh,piwik.ambau.ovh,plumxml.ambau.ovh,pma.ambau.ovh,roundcube.ambau.ovh,shaarli.ambau.ovh,snippet.ambau.ovh,social.ambau.ovh,st.ambau.ovh,statistiques.ambau.ovh,stats.ambau.ovh,test.ambau.ovh,tiddly.ambau.ovh,tuleap.ambau.ovh,vma.ambau.ovh,vmail.ambau.ovh,webmail.ambau.ovh,wiki.ambau.ovh,www.ambau.ovh,zabbix.ambau.ovh"

domains_ggj_fr="ggj.fr,ada.ggj.fr,admin.ggj.fr,blog.ggj.fr,carte.ggj.fr,chat.ggj.fr,chiffres.ggj.fr,cloud.ggj.fr,cms.ggj.fr,code.ggj.fr,compta.ggj.fr,datadoghq.ggj.fr,democratie.ggj.fr,dev.ggj.fr,diaspora.ggj.fr,discord.ggj.fr,discourse.ggj.fr,down.ggj.fr,elastik.ggj.fr,fil.ggj.fr,filerun.ggj.fr,files.ggj.fr,forum.ggj.fr,gestion.ggj.fr,git.ggj.fr,gitea.ggj.fr,gitlab.ggj.fr,goaccess.ggj.fr,grav.ggj.fr,graylog.ggj.fr,info.ggj.fr,kibana.ggj.fr,lcr.ggj.fr,liens.ggj.fr,links.ggj.fr,log.ggj.fr,manage.ggj.fr,matomo.ggj.fr,metrics.ggj.fr,monitor.ggj.fr,nextcloud.ggj.fr,pfa.ggj.fr,piwik.ggj.fr,plumxml.ggj.fr,pma.ggj.fr,roundcube.ggj.fr,shaarli.ggj.fr,snippet.ggj.fr,social.ggj.fr,st.ggj.fr,statistiques.ggj.fr,stats.ggj.fr,test.ggj.fr,tiddly.ggj.fr,tuleap.ggj.fr,vma.ggj.fr,vmail.ggj.fr,webmail.ggj.fr,wiki.ggj.fr,www.ggj.fr,zabbix.ggj.fr"

domains_ggj_ovh="ggj.ovh,ada.ggj.ovh,admin.ggj.ovh,blog.ggj.ovh,carte.ggj.ovh,chat.ggj.ovh,chiffres.ggj.ovh,cloud.ggj.ovh,cms.ggj.ovh,code.ggj.ovh,compta.ggj.ovh,datadoghq.ggj.ovh,democratie.ggj.ovh,dev.ggj.ovh,diaspora.ggj.ovh,discord.ggj.ovh,discourse.ggj.ovh,down.ggj.ovh,elastik.ggj.ovh,fil.ggj.ovh,filerun.ggj.ovh,files.ggj.ovh,forum.ggj.ovh,gestion.ggj.ovh,git.ggj.ovh,gitea.ggj.ovh,gitlab.ggj.ovh,goaccess.ggj.ovh,grav.ggj.ovh,graylog.ggj.ovh,info.ggj.ovh,kibana.ggj.ovh,lcr.ggj.ovh,liens.ggj.ovh,links.ggj.ovh,log.ggj.ovh,manage.ggj.ovh,matomo.ggj.ovh,metrics.ggj.ovh,monitor.ggj.ovh,nextcloud.ggj.ovh,pfa.ggj.ovh,piwik.ggj.ovh,plumxml.ggj.ovh,pma.ggj.ovh,roundcube.ggj.ovh,shaarli.ggj.ovh,snippet.ggj.ovh,social.ggj.ovh,st.ggj.ovh,statistiques.ggj.ovh,stats.ggj.ovh,test.ggj.ovh,tiddly.ggj.ovh,tuleap.ggj.ovh,vma.ggj.ovh,vmail.ggj.ovh,webmail.ggj.ovh,wiki.ggj.ovh,www.ggj.ovh,zabbix.ggj.ovh"

domains_otokoz_ovh="otokoz.ovh,ada.otokoz.ovh,admin.otokoz.ovh,blog.otokoz.ovh,carte.otokoz.ovh,chat.otokoz.ovh,chiffres.otokoz.ovh,cloud.otokoz.ovh,cms.otokoz.ovh,code.otokoz.ovh,compta.otokoz.ovh,datadoghq.otokoz.ovh,democratie.otokoz.ovh,dev.otokoz.ovh,diaspora.otokoz.ovh,discord.otokoz.ovh,discourse.otokoz.ovh,down.otokoz.ovh,elastik.otokoz.ovh,fil.otokoz.ovh,filerun.otokoz.ovh,files.otokoz.ovh,forum.otokoz.ovh,gestion.otokoz.ovh,git.otokoz.ovh,gitea.otokoz.ovh,gitlab.otokoz.ovh,goaccess.otokoz.ovh,grav.otokoz.ovh,graylog.otokoz.ovh,info.otokoz.ovh,kibana.otokoz.ovh,lcr.otokoz.ovh,liens.otokoz.ovh,links.otokoz.ovh,log.otokoz.ovh,manage.otokoz.ovh,matomo.otokoz.ovh,metrics.otokoz.ovh,monitor.otokoz.ovh,nextcloud.otokoz.ovh,pfa.otokoz.ovh,piwik.otokoz.ovh,plumxml.otokoz.ovh,pma.otokoz.ovh,roundcube.otokoz.ovh,shaarli.otokoz.ovh,snippet.otokoz.ovh,social.otokoz.ovh,st.otokoz.ovh,statistiques.otokoz.ovh,stats.otokoz.ovh,test.otokoz.ovh,tiddly.otokoz.ovh,tuleap.otokoz.ovh,vma.otokoz.ovh,vmail.otokoz.ovh,webmail.otokoz.ovh,wiki.otokoz.ovh,www.otokoz.ovh,zabbix.otokoz.ovh"

domains_coworking-lannion_org="cloud.coworking-lannion.org"
#domains="coworking-lannion.org"

domains="17112018.fr 17112018.ovh ambau.ovh ggj.fr ggj.ovh otokoz.ovh";

CLEAN

Clean one domain

domain="17112018.fr"
files="/etc/letsencrypt/archive/${domain} /etc/letsencrypt/csr/${domain} /etc/letsencrypt/keys/${domain} /etc/letsencrypt/live/${domain} /etc/letsencrypt/renewal/${domain}.conf"

for file in $files; do [ -e "$file" ] && rm -fR "$file"; done

Clean all domains

files="/etc/letsencrypt/archive/* /etc/letsencrypt/csr/* /etc/letsencrypt/keys/* /etc/letsencrypt/live/* /etc/letsencrypt/renewal/*"

for file in $files; do [ -e "$file" ] && rm -fR "$file"; done

CERBOT

stop services OpenVZ

ctids_web="$(vzlist -h "*-php*" -Ho ctid|xargs) $(vzlist -h "*-apache*" -Ho ctid|xargs)"
netstat -lnt | grep ':80'

# stop port 80
service haproxy stop
vz-launch -y 'systemctl stop apache2.service' $ctids_web
#vz-launch -y 'duniter stop' 189
netstat -lnt | grep ':80'

create new domain

for domain in ${domains}; do
    subdomains=domains_${domain//./_}
    echo "-- ${domain} --"
    certbot certonly --standalone -d ${!subdomains}
    cat /etc/letsencrypt/live/${domain}/fullchain.pem /etc/letsencrypt/live/${domain}/privkey.pem > /etc/server/ssl/private/letsencrypt-${domain}.pem
done

renew domains

file="/etc/server/ssl/private/letsencrypt.pem.lst"
domains=$(ls /etc/letsencrypt/renewal/|sed 's|\.conf||g')
certbot renew
[ -f "$file" ] && mv "$file" "$file.keep$(date +%s)"
for domain in ${domains}; do
    file_pem="/etc/server/ssl/private/letsencrypt-${domain}.pem"
    cat /etc/letsencrypt/live/${domain}/fullchain.pem /etc/letsencrypt/live/${domain}/privkey.pem > "$file_pem"
    echo "${file_pem}" >> "${file}"
done

start services

# start port 80
vz-launch -y 'systemctl start apache2.service' $ctids_web
#vz-launch -y 'duniter webstart --webmhost 10.0.0.189' 189
service haproxy start
netstat -lnt | grep ':80'

bash · haproxy · letsencrypt

Fri Nov 2 19:16:17 2018 * · permalink

http://code.ambau.ovh/snippets/letsencrypt.html

mooc - python - 5 - module

https://docs.python.org/3/reference/simple_stmts.html#the-import-statement

module are:

are mutable !!
define a namespace
is a .py file
the file are searched first in sys.path (after in local path)
import are only made on time during a program

IMPORT

import <module>

import as
absolute import
the file are searched first in sys.path (after in local path)

# module_name is a string, it can be a variable
from importlib import module_name
module_othername = module_name('ma'+'th')

# alias
import module_name as module_othername

RELATIVE IMPORT

local path first (before look in sys.path)
import global module with namespace reference
abstract the calling !!

import a module

from . import module_name as module_othername

import just an object from module

# import selected object in module without entire module
from .module_name import object_name as object_othername

local parent path first
import global module with namespace reference

import a module

from .. import module_name as module_othername

import just an object from module

# import selected object in module without entire module
from ..module_name import object_name as object_othername

ENTRY POINT

for the entry point of program, _name__ return main and not the filename !
So you can find in the main file

if __name__ == "__main__":
....

Let program make specific codes when apply file is calling directly from interpreter & not from an import when calling a file directly from the interpreter, the entry point sets name to 'main', so it s almost impossible to use relative imports in this situation

To use test envirronement:

python3 -m unittest package_name.module_name

http://sametmax.com/un-gros-guide-bien-gras-sur-les-tests-unitaires-en-python-partie-1/

IMPORTLIB

Use a variable for module name

from importlib import import_module
modulename = "ma" + "th"
loaded = import_module(modulename)
math is loaded

Import math <=> math = import_module('math')

from pathlib import Path
<=>
tmp = import_module('pathlib')
Path = tmp.Path
del tmp

IMPORT / FROM ... IMPORT

import

import all objects from module
complete isolation
access to all objects in module with notation MODULE.OBJECT

From ... import

import only selected object
no isolation & iomportation of object in current namespace
no access to other object in module

RELOAD

Force to reload manualy a module

from importlib import reload
reload(MODULE)

AUTORELOAD

Load magic 'autoreload'

%load_ext autoreload

Enable autoreload

%autoreload 2

ATTRIBUTES

https://docs.python.org/3/tutorial/modules.html#importing-from-a-package

__name__

Module name

__file__

The complete path of package definition file: init.py

__all__

redefine the behavior of import *

PATH

https://docs.python.org/3/tutorial/modules.html#the-module-search-path

Environnement variable

    import os
    os.getcwd() # return current working directory
    os.listdir(path) # return the list of path & files in path (if empty return current path)
    os.environ # dict of full os informations
    os.environ['PATH'] os.environ['PYTHONPATH']
    sorted([i for i in os.environ])

Include paths

import sys
sys.path

Add path when launch python script

PYTHONPATH=/path python script.py

SYS.MODULES

https://docs.python.org/3/library/sys.html#sys.modules

example

import sys

# test on loaded modules
'csv' in sys.modules
import csv
'csv' in sys.modules
csv is sys.modules['csv']

# unload module
del sys.modules['multiple_import']

# A tuple of strings giving the names of all modules that are compiled into this Python interpreter
sys.builtin_module_names

# modules.keys() only lists the imported modules
sys.modules.keys()
'csv' in sys.modules.keys()

CWD

from pathlib import Path
Path.cwd()

IMPORT FILE FROM ANYWHERE

from pathlib import Path
directory_installation = Path(__file__).parent
import sys
sys.path.append(directory_installation)

MODTOOLS

Show_module

from modtools import show_module
import math
show_module(math)

SETUPTOOLS / PYPI

https://pypi.org/project/setuptools/
https://pypi.org/

Deploy setuptools

pip3 install setuptools

PACKAGE

https://docs.python.org/3/tutorial/modules.html#packages

A package are:

a folder with the same name
a namespace
this folder have to contains a mandatory file init.py

import package

import package_name
import package_name as package_othername

Use namespace

package_name.object

not recommended

package_name.module_name.object

Subpackage
Package physically inside another

import package_name.subpackage_name
import package_name.subpackage_name.module_name.element_name

EXCERCISE

import this
def decode_zen(this):
    return ''.join(this.d[i] if i in this.d else i for i in this.s)

# better
def decode_zen(this):
    return ''.join(this.d.get(i) for i in this.s)

mooc · python

Sat Oct 27 12:33:06 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-5-module.html

mooc - python - 5 - lambda

LAMBDA

Anonym function, to use as an expression

from functools import reduce
from operator import add
def factoriel(x):
    return reduce(lambda x, y: x*y, range(1, x+1), 1)
def fibonacci(x):
    return reduce(operator.add, range(1, x+1), 0)
def fibonacci(x):
    return reduce(add, range(1, x+1), 0)

With comprehension
[(lambda x: x**2)(x) for x in range(10) if x % 2 == 0]

MAP

Return an iterator that applies function to every item of iterable, yielding the results

list(map(lambda x: x**2, filter(lambda x: x % 2 == 0, range(10))))

FILTER

Construct an iterator from those elements of iterable for which function returns true

list(filter(lambda x: x % 2 == 0, range(10)))

EXERCISE

1-1

More efficient, don't duplicate listes

def multi_tri(listes):
    for liste in listes:
        liste.sort()
    return listes
def multi_tri(listes):
    return list(map(sorted, listes))

1-2

More efficient, don't duplicate listes

def multi_tri_reverse(listes, reverses):
    for x,y in zip(listes, reverses):
        x.sort(reverse=y)
    return listes
def multi_tri_reverse(listes, reverses):
    return [sorted(x, reverse=y) for x,y in zip(listes, reverses)]
def multi_tri_reverse(listes, reverses):
    return list(map(lambda x, y: sorted(x, reverse=y), listes, reverses))

2-1

def doubler_premier(f, arg, *args):
    return f(2*arg,*args)
def doubler_premier(f, arg1, *args):
    from functools import reduce
    from operator import add, mul
    return reduce(
        f,
        args + (arg1*2,),
        1 if f is mul else 0
    )

2-2

def doubler_premier_kwds(f, arg, *args, **kwargs):
    return f(2*arg, *args, **kwargs)
def doubler_premier_kwds(f, arg1, *args, y=0, z=0):
    from functools import reduce
    return reduce(
        f,
        [arg for arg in args + (arg1*2,) + (y,) + (z,) if arg],
        1 if f.__name__ == 'mul3' else 0
    )

3-1

def compare_all(f, g, entrees):
    return [f(entree) == g(entree) for entree in entrees]

3-2

def compare_args(f, g, entrees):
    return [f(*entree) == g(*entree) for entree in entrees]

mooc · python

Sat Oct 27 12:32:11 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-5-lambda.html

mooc - python - 5 - iter

https://docs.python.org/fr/3.7/howto/functional.html
https://docs.python.org/3/library/itertools.html

ITERATOR

iterator allow to calculate data of function only on read it
builtin function all methods on object

s = {0, 1, 'string1', 3, 'string2',5 , 6}
[x for x in s if type(x) is int]
i = iter(s)
type(i)
dir(i)
next(i)
i.__next__()

ITERABLE

Object contains a method 'iter', which gives iterator methods
An object wich on you can use iterator methods

ITERTOOLS

https://docs.python.org/3/library/itertools.html

This module implements a number of iterator building blocks

import itertools

Infinite iterators

count(start, [step])

start, start+step, start+2*step, …
count(10) --> 10 11 12 13 14 ...

cycle(p)

p0, p1, … plast, p0, p1, …
cycle('ABCD') --> A B C D A B C D ...

repeat(elem [,n])

elem, … endlessly or up to n times
repeat(10, 3) --> 10 10 10

Iterators terminating on the shortest input sequence

accumulate(p[,func])

p0, p0+p1, p0+p1+p2, …
accumulate([1,2,3,4,5]) --> 1 3 6 10 15

chain(p, q, …)

p0, p1, … plast, q0, q1, …
chain('ABC', 'DEF') --> A B C D E F

chain.from_iterable(iterable)

p0, p1, … plast, q0, q1, …
chain.from_iterable(['ABC', 'DEF']) --> A B C D E F

compress(data, selectors)

(d[0] if s[0]), (d[1] if s[1]), …
compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F

dropwhile(pred, seq)

seq[n], seq[n+1], starting when pred fails
dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1

filterfalse(pred, seq)

elements of seq where pred(elem) is false
filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8

groupby(iterable[, keyfunc])

sub-iterators grouped by value of keyfunc(v)

islice(seq, [start,] stop [, step])

elements from seq[start:stop:step]
islice('ABCDEFG', 2, None) --> C D E F G

starmap(func, seq)

func(seq[0]), func(seq[1]), …
starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000

takewhile(pred, seq)

seq[0], seq[1], until pred fails
takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4

tee(it, n)

it1, it2, … itn splits one iterator into n

zip_longest(p, q, …)

(p[0], q[0]), (p[1], q[1]), …
zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-

Combinatoric iterators

product(p, q, … [repeat=1])

cartesian product, equivalent to a nested for-loop
product('ABCD', repeat=2) -> AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD

permutations(p[, r])

r-length tuples, all possible orderings, no repeated elements
permutations('ABCD', 2) -> AB AC AD BA BC BD CA CB CD DA DB DC

combinations(p, r)

r-length tuples, in sorted order, no repeated elements
combinations('ABCD', 2) -> AB AC AD BC BD CD

combinations_with_replacement(p, r)

r-length tuples, in sorted order, with repeated elements
combinations_with_replacement('ABCD', 2) -> AA AB AC AD BB BC BD CC CD DD

mooc · python

Sat Oct 27 12:31:35 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-5-iter.html

mooc - python - 5 - generator

EXPRESSION

https://docs.python.org/fr/3.7/howto/functional.html

create an generator of iterator in place of comprehension

(treatment for ... if ...)

carre = (x**2 for x in range(1000))
palin = (x for x in carre if str(x) == str(x)[::-1])
carre
palin
# only now, calculate
list(palin)

GENERATOR FUNCTION

http://python-history.blogspot.fr/2010/06/from-list-comprehensions-to-generator.html

def carre(a, b):
    for i in range(a, b):
        yield i**2

def palin(it):
    for i in it:
        if (isinstance(i, (str, int)) and
            str(i) == str(i)[::-1]):
            yield i

list(palin(x**2 for x in range(1000)))

EXCERCISE

def produit_scalaire(X, Y):
    return sum(x*y for x, y in zip(X, Y))
g = produit_scalaire((1, 2), (3, 4))

mooc · python

Sat Oct 27 12:29:19 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-5-generator.html

mooc - python - 5 comprehension

https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

ages = [10, 120, 110, 25, 30, 40, 50]
l1 = [a+1 if a < 100 else a for a in ages]
l2 = [a+1 for a in ages if a < 100]

ages = {'ana':20, 'EVE':30, 'bob':40}
prenoms = [p.lower() for p, a in ages.items()]

SET

prenoms = ['ana', 'eve', 'ALICE', 'Anne', 'bob']
s = {p.lower() for p in prenoms if p.lower().startswith('a')}

DICT

ages = {'ana':20, 'EVE':30, 'bob':40}
d = {p.lower():a + 1 for p, a in ages.items()}

prenoms = ['ana', 'eve', 'ALICE', 'Anne', 'bob']
ages = [10, 20, 30, 40, 50]
d = {p.lower():a + 1 for p, a in zip(prenoms, ages)}

# Some troubles with mixed list & set
prenoms = ['ana', 'eve', 'ALICE', 'Anne', 'bob']
ages = [10, 20, 30, 40, 50]
list(zip(prenoms, ages))

NESTING

flat nesting : 1 level => [n(p(..., ...) ,...)]

[... for ... for ...]
[... >   ... >  ...]
for n ...
  for p ...
    ...
[str(n) + str(p) for n in [2, 4] for p in [10, 20, 30]]

classic "sub"

[... for n in [... p ...] if ...]
for p ...
for n ...

embedded "sub" nesting 2 levels => [p[n... ], ...]

[[... for ...] for ...]
[[... <   ...] <   ...]
for p ...
  for n ...
[[str(n) + str(p) for n in [2, 4]] for p in [10, 20, 30]]

FLAT

[item for sublist in list for item in sublist]
for sublist in list
  for item in sublist
    item

REPEAT

# compare:
[x + y for x in (1,2,3) for y in (10,20) for z in (0,2)]
[x + y for x in (1,2,3) for z in (0,2) for y in (10,20)]
[x + y for x in (1,2,3) for z in (0,2) for y in (10,20)]

t = (y for y in (x**2 for x in range(1000)) if str(y) == str(y)[::-1])

with restriction

[str(n) + str(p) for n in [2, 4] for p in [10, 20, 30] if n*p >= 40]

BREAK

def end_of_loop():
    raise StopIteration
even = list(end_of_loop() if n == 412 else n for n in numbers if 0 == n % 2)

even = list(next(iter(())) if n == 412 else n for n in numbers if 0 == n % 2)

EXERCISE

1

# flat vs nested
[[str(n) + str(p) for n in (2, 4)] for p in (10, 30, 50)]
[str(n) + str(p) for p in (10, 30, 50) for n in (2, 4)]

# flat vs nested
[[str(n) + str(p) for n in (2, 4)] for p in (10, 30, 50)]
[str(n) + str(p) for p in (10, 30, 50) for n in (2, 4)]

def aplatir(conteneurs):
    return [c2 for c1 in conteneurs for c2 in c1]
def aplatir(conteneurs):
    r=[]
    for c1 in conteneurs:
        for c2 in c1:
             r.append(c2)
    return r
aplatir(((1, [2, 3]), [], 'a', ['b', 'c']))

2

def alternat(c1, c2):
    return [c_2 for c_1 in zip(c1, c2) for c_2 in c_1]
alternat( (1, 2, 3), ('a', 'b', 'c'))

3

def intersect(A, B):
    return {c2 for c1 in ((v1, v2) for i1, v1 in A for i2, v2 in B if i1 == i2) for c2 in c1}
intersect({ (1, 'unA'), (2, 'deux'), (3, 'troisA')}, { (1, 'unB'), (2, 'deux'), (4, 'quatreB')})
# more elegante
def intersect(A, B):
    return {v for dictionnaire in [dict(A),dict(B)] for k,v in dictionnaire.items() if k in dict(A) and k in dict(B)}
def intersect(A, B):
    return { i for item in (dict(A).keys() & dict(B).keys()) for i in (dict(A)[item], dict(B)[item]) }

mooc · python

Sat Oct 27 12:27:54 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-5-comprehension.html

phpquery

print informations about php version & modules

 -v version -s sapi_name -M 
 -V # list all available versions of php client
 -v $version -S # list all available sapi
 -v $version -S $sapi -M # list all available modules
 -v $version -s $sapi -m $module # return informations about module $module

tests the availability of the module

phpquery -q -v $version -s $sapi -m $module && echo YES

bash · php

Fri Oct 19 23:44:11 2018 * · permalink

http://phpquery

mooc - python - 4 - variable

SCOPE

https://www.programiz.com/python-programming/namespace

Order to interprate variables is LEGB

Local
Enclosing
Global
Builtin

locals()

return a list of actual local scope variables

globals()

return a list of actual global scope variables

Global scope & function error !!

Because var is referenced like global by "print(var)" & use like local with "var = 'local'"
Solution: remove "print(var)" or remove "var = 'local'" or put it after "var = 'local'"

var = 'global'
def f():
    print(var)
    var = 'local'
f()
print(var)

GLOBAL

to use global variable locally

var = 'global'
def f():
    global var
    var = 'local'
    print(var)
f()
print(var)

NONLOCAL

To use upper local scope variable locally
The upper scope local variable is the closest one
A nonlocal variable have to be a binding variable in upper local scope (in function definition, not in the global scope) else an exception are called

a = 'a global'
def f():
    a = 'a f'
    def g():
        nonlocal a
        a = 'a g'
        print(a)
    g()
    print(a)
f()
print(a)

BUILTINS

import builtins
modules with all builtin functions

dir(builtins)
dir(builtins.str)

To ensure to call a really builtin function
import builtins

with builtins.open(...) as f:

GLOBAL VARIABLES

vars()
globals()

To access object to other namespace

print(dir(spam))
print(dir(spam.__builtins__))

mooc · python

Fri Oct 19 20:16:04 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-4-variable.html

mooc - python - 4 - test

GENERAL

https://docs.python.org/3/library/stdtypes.html#truth-value-testing
https://docs.python.org/3/reference/expressions.html#conditional-expressions
http://legacy.python.org/dev/peps/pep-0308/
https://docs.python.org/3/tutorial/datastructures.html#more-on-conditions

Order to return test:

call bool
call len

bool(object)

object.bool()
object.len()

type built-in
false : False None [] {} () ''
true : others

INSTRUCTION / EXPRESSION

For if you can use any of expressions
Instructions Expressions
assignment function call
import operators is, in, ==, ...
instruction if conditional expression
instruction for List Comprehensions

syntax

if condition:
...
elif condition:
...
else:
...

inputs = [23, 65, 24]
def condition(n):
    return (n**2) % 10 == 5

if [value for value in inputs if condition(value)]:
    print("au moins une entrée convient")

FEW OPERATORS TEST IN LINE

if 0 < a < 10 < b:
    instruction

TERNARY OPERATORS & SHORT TEST

(result_if_false, result_if_true)[condition]

Result_if_true if condition else result_if_false # ternary operators

is_nice = True

if is_nice:
    state = "nice"
else:
    state = "not nice"
<=>
# ternary operators
state = "nice" if is_nice else "not nice"
<=>
# short test
state = ("not nice", "nice")[is_nice]

print(state)

SHORT-CIRCUIT

if the first tests are sufficient the following ones are not evaluated

# stop at first test
false and true
true or false

NESTING

valeur = -1 if x < -10 else (0 if x <= 10 else 1)

BOOL

Use bool function to test the behavior of return

def show_bool(x):
    print(f"condition {repr(x):>10} considérée comme {bool(x)}")

for exp in [None, "", 'a', [], [1], (), (1, 2), {}, {'a': 1}, set(), {1}]:
    show_bool(exp)

OPERATORS

| Famille | |Exemples | |---------|-|---------| | Égalité | |==, !=, is, is not | | Appartenance | | in | | Comparaison | | <=, <, >, >= | | Logiques | | and, or, not |

PRIORITY ORDER

a and not b or c and d
<=>
(a and (not b)) or (c and d)

Operators can be used with others than boolean
The last non evaluated elements is always return else when the short-circuit can be used
1 and [1, 2]
1 or [1, 2]
1 and 2 and 3
1 and 2 and 3 and '' and 4
[] or "" or {}
[] or "" or {} or 4 or set()

EXCERCISE

1

def dispatch1(a, b):
    def isimpair(x): return(x%2)
    def ispair(x): return(not x%2)
    if ispair(a) and ispair(b): return(a**2 + b**2)
    elif ispair(a) and isimpair(b): return(a * (b - 1))
    elif isimpair(a) and ispair(b): return((a - 1) * b)
    else: return(a**2 - b**2)

2

def dispatch2(a, b, A, B):
    if a in A and not (b in B): return(a * (b - 1))
    elif not (a in A) and b in B: return((a - 1) * b)
    else: return(a**2 + b**2)

3

def libelle(ligne):
    for i in [" ", "\t"]:
        ligne = ligne.replace(i, '')
    ligne = ligne.split(',')
    if len(ligne) != 3:
        return
    prenom, nom, rang = ligne
    if not rang: rang = "-ème"
    elif int(rang) == 1: rang = "1er"
    else: rang = f"{rang}-ème"
    return(f"{nom}.{prenom} ({rang})")

mooc · python

Fri Oct 19 20:15:50 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-4-test.html

mooc - python - 4 - loop

WHILE

Execute the loop while condition is true and execute else:

while condition:
    ...
else:
    ...

continue

Step in a loop to the next loop

break

Break the loop definitively

while true

while true, continual loop with a break to exit

while true:
    ...
    break

example

while liste:
element = liste.pop()
print(element)

FOR

for item in element
    ...
else
    ...

EXCERCISE

1

def pgcd(a, b):
    if a == 0:
        return(b)
    elif b == 0:
        return(a)
    while True:
        if b > a:
            b = b % a
            if b == 0:
                return(a)
        else:
            a = a % b
            if a == 0:
                return(b)

2

def taxes(income):
    taxe = 0
    if income > 150_000:
        taxe += (income - 150_000) * (45/100)
        income = 150_000
    if income > 45_000:
        taxe += (income - 45_000) * (40/100)
        income = 45_000
    if income > 11_500:
        taxe += (income - 11_500) * (20/100)
        income = 11_500
    return (int(taxe))

2 bis

def taxes(income):
    taxe = 0
    limit_rates = (
        (150_000, 45),
        (45_000, 40),
        (11_500, 20)
    )
    for limit, rate in limit_rates:
        if income > limit:
            taxe += (income - limit) * (rate/100)
            income = limit
    return (int(taxe))

mooc · python

Fri Oct 19 20:15:25 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-4-loop.html

mooc - python - 4 - function

A function performs a variable passing by reference !!

class Test:
    """
    class Test to make a test
    with a long text to see differtence
    """
    def __init__(self):
        """init property listin with an empty list"""
        self.listin = []
    def add(self, value):
        """add value to listin property"""
        self.listin.append(value)
    def print(self):
        "print listin property to stdout"
        print(self.listin)

a = [0]
p.add(a)
p.print()
a[0] = 100
p.print()

DOC

https://legacy.python.org/dev/peps/pep-0257/

help(Class)

Complet help for Test class

Class?

Short help, for ipython

Class.doc

Progamming help

Class.add?

Method help, ipython

TOOLS

isinstance(type)

Return boolean & support class inheritance

class Animal:
    def __init__(self, name):
        self.name = name

class Mammifere(Animal):
    def __init__(self, name):
        Animal.__init__(self, name)
isinstance(baleine, Mammifere)
isinstance(baleine, Animal)

types # constants types

from types import FunctionType
isinstance(factoriel, FunctionType)
isinstance(len, BuiltinFunctionType)

# list all types
import types
dir(types)

TYPE HINTS

https://docs.python.org/3/library/typing.html

Since python 3.5, allow optionnal type hinting

var : int = 0
def fact(n : int) -> int:

List

def foo(x: List[int]) -> List[str]:

Iterable

def lower_split(sep: str, inputs : Iterable[str]) -> str:

Multi typing

from typing import Dict, Tuple, List
ConnectionOptions = Dict[str, str]
Address = Tuple[str, int]
Server = Tuple[Address, ConnectionOptions]
def broadcast_message(message: str, servers: List[Server]) -> None:
    ...

https://docs.python.org/3/library/typing.html#user-defined-generic-types

Extension

from typing import NewType
UserId = NewType('UserId', int)
user1_id : UserId = 0

example

from typing import TypeVar, Generic
from logging import Logger
T = TypeVar('T')

class LoggedVar(Generic[T]):
    def __init__(self, value: T, name: str, logger: Logger) -> None:
        self.name = name
        self.logger = logger
        self.value = value

    def set(self, new: T) -> None:
        self.log('Set ' + repr(self.value))
        self.value = new

    def get(self) -> T:
        self.log('Get ' + repr(self.value))
        return self.value

    def log(self, message: str) -> None:
        self.logger.info('%s: %s', self.name, message)

mooc · python

Fri Oct 19 20:14:44 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-4-function.html

mooc - python - 4 - function: parameter & argument

GENERAL

Order of parameters

ordered
named
*tuple
**dict

ORDERED

Ordered (mandatory) arguments have to be in first in list of arguments

def f(arg1, arg2, ...):
    ...

default/optionnal parameters

Default parameters have to be placed in last in list of parameters

def f(arg1, arg2, arg3=opt, arg4=opt, ...):
    ...

NAMED

You can give parameters to the function with naming

def f(arg1, arg2, arg3):
    ...
f(arg2=data, arg3=data, arg1=data)

TUPLE

def f(*t):
    ...
def f(arg1, arg2, *t):
    print(f"arg1: {arg1}\narg2: {arg2}\nothers: {t}")

f('nom', 'prenom', list(range(5)), 'et moi', ['etc'])

With optionnal parameters

def f(nom, prenom='henriette', *d):
    print(f"nom: {nom}\nprenom: {prenom}\nd: {d}")

f('louis', 'mom', 20, 'pierre', ['tyt'])

DICTIONNARY

def f(**d):
    print(f"d: {d}")

f(tata='mom', tonton='pierre', maman='tyt')

With optionnal parameters

def f(nom, prenom='henriette', **d):
    print(10*"=" + f"\nnom: {nom}\nprenom: {prenom}\nd: {d}")

f('louis', 'prenom', tata='mom', tonton='pierre', maman='tyt')
f('louis', tata='mom', tonton='pierre', maman='tyt')

UNPACKING

Used to send arguments to a function from a list / tuple / set
With a good lenght of arguments !!

def f(a, b):
    print(10*"-" + f"\na: {a}\nb: {b}")

l = [10, 'toto']
f(*l)
l = (10, 'toto')
f(*l)
l = set([10, 'toto']);
f(*l)

DICTIONNARY SENDING

Use a dictionnary to send arguments to function
Keys names of dictionnary have to be match with arguments names

def f(a, b):
    print(10*"-" + f"\na: {a}\nb: {b}")
{'a': 10, 'b': 'toto'}
f(**d)

Use for print (& wrapping)

pp = {'sep':'-', 'end':' >EOL'}
print(10, 20, **pp)

TRAP

named argument & tuple argument

https://docs.python.org/3/reference/expressions.html#calls

def foo(a, b, c, d):
    print(a, b, c, d)

#good
    foo(1, c=3, *(2,), **{'d':4})

# wrong
foo (1, b=3, *(2,), **{'d':4})

don t use mutable object in default argument

https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects

import random
def ajouter_un_aleatoire(resultats=[]):
    resultats.append(random.randint(0, 10))
    return resultats
ajouter_un_aleatoire()
ajouter_un_aleatoire()

EXCERCISE

1

def distance(*arg):
    from math import sqrt
    result = sqrt(sum([i**2 for i in arg]))
    return result if result != 0 else 0

2

def numbers(*args):
    return (sum(args), min(args, default=0), max(args, default=0))

mooc · python

Fri Oct 19 20:14:12 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-4-function-parameter-argument.html

mooc - python - 3 - tuple

https://www.python.org/dev/peps/pep-3132/

assignment

tuple1 = 1, 2
tuple2 = (1, 2)
tuple3 = 1, 2,
tuple4 = (1, 2,)

Assignment of one element

simple1 = 1,
simple2 = (1,)

Pretty print

tuple1 = (
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9],
)

addition

tuple3 = tuple1 + tuple2
tuple1 += (3, 4,)

cast

tuple = tuple(list)

SEQUENCE UNPACKING

En réalité, les seules contraintes fixées par Python sont que:

Le terme à droite du signe = soit un itérable (tuple, liste, string, etc.) ;
Le terme à gauche soit écrit comme un tuple ou une liste - notons tout de même que l'utilisation d'une liste à gauche est rare et peu pythonique ;
Les deux termes aient la même longueur - en tout cas avec les concepts que l'on a vus jusqu'ici, mais voir aussi plus bas l'utilisation de *arg avec le extended unpacking.

Switch variables

a, b = b, a

EXTENDED UNPACKING

1

reference = [1, 2, 3, 4, 5]
a, *b, c = reference
print(f"a={a} b={b} c={c}")
a, b, *_ = reference # '_' is for tell to reader that the following values are not used
print(f"a={a} b={b} c={c}")

2

reference = [1, 2, 3]
_, milieu, _ = reference
print('milieu', milieu)

3

ignored, ignored, right = reference
print('right', right)

Deep

structure = ['abc', [(1, 2), ([3], 4)], 5]
(a, (b, ((trois,), c)), d) = structure
print('trois', trois)
(a, (b, ([trois], c)), d) = structure
print('trois', trois)

Ultra deep

Un exemple très alambiqué avec plusieurs variables *extended

tree = [1, 2, [(3, 33, 'three', 'thirty-three')], ( [4, 44, ('forty', 'forty-four')])]
*_,  ((_, *x3, _),), (*_, x4) = tree
print(f"x3={x3}, x4={x4}")

# for
l = [(1, 2), (3, 4), (5, 6)]
for a, b in l:
    print(f"a={a} b={b}")

for i, j, k in zip(range(3), range(100, 103), range(200, 203)):
    print(f"i={i} j={j} k={k}")

enumerate()

Return an enumerate object with iterator in index and value as value

villes = ["Paris", "Nice", "Lyon"]
for i, ville in enumerate(villes):
    print(i, ville)

EXERCISE

1

def comptage(in_filename, out_filename):
    import re
    with open(in_filename, "r", encoding='utf-8') as f_in:
        with open(out_filename, "w", encoding='utf-8') as f_out:
            i = 0
            for line_in in f_in:
                i += 1
                # wc = len(re.findall('(\w|\.\.\.+)+',line_in))
                wc = len(line_in.split())
                cc = len(line_in)
                f_out.write(f"{i}:{wc}:{cc}:{line_in}")

2

def surgery(liste):
    l=len(liste)
    if l in [0, 1]: pass
    elif l%2 == 0: liste[1], liste[0] = liste[0], liste[1]
    elif l%2 == 1: liste[-2], liste[-1] = liste[-1], liste[-2]
    return(liste)

mooc · python

Fri Oct 19 20:13:53 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-3-tuple.html

mooc - python - 3 - set

https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset

Usefull to stock only uniq elements
Set is mutable but element in set ar to be immutable => unable to create a set of set
Like distionnary, set are not ordered

INIT

s = set()
s = {key1, key2, ...}
s = set([key1, key2, ...])

METHODS

(key) in s

Return true in key is in s

len(s)

Print length of s

s.clear()

Clear elements of s

s.add()

Add a key in set

s.update(s2)

Merge a set into a set

s.discard(key)

Remove elements, no error if not found

s.remove(key)

Remove elements, launch exception if not found

try:
    s.remove(key)
except KeyError as e:
    print("text", e)

s.pop()

Remove last or indexed element but there is no order in a set !

s = [1, 2, 3, 4, 5]

while s:
    s = s.pop()
    print("set", s)
print("set is now empty", s)

OPERATIONS

union

s1 | s2
Return the union of sets

intersection

s1 & s2
Return intersection of sets

difference

s2 - s3
Return difference of two sets

symmetrical difference

s1 ^ s2
AΔB=(A−B)∪(B−A)=(A∪B)-(A∩B)

equality

s1 == s2

inclusion

s1 <= s2
s1 < s2

Disjoint sets

s.isdisjoint(s2)

FROZENSET

Immutable set to serve to key of dictionnary or element of set

init

fs = frozenset()
fs = frozenset([key1, key2, ...])

Available methods

copy()
isdisjoint()
symmetric_difference()
difference()
issubset()
union()
intersection()
issuperset()

EXCERCISE

1

def read_set(filename):
    s=set()
    with open(filename, "r", encoding='utf-8') as f:
        for line in f:
            #print(line)
            s.update({line.strip()})
    return(s)

2

def search_in_set(filename_reference, filename):
    s = set()
    l = []
    with open(filename_reference, "r", encoding='utf-8') as f_ref:
        for line in f_ref:
            s.update({line.strip()})
    with open(filename, "r", encoding='utf-8') as f_test:
        for line in f_test:
            line = line.strip()
            l.append((line, line in s))
    return(l)

3

def diff(extended, abbreviated):
    # get ships
    ships_ext = set([i[0] for i in extended])
    ships_abb = set([i[0] for i in abbreviated])
    ships_name = {i[0]: i[4] for i in extended}
    # get sets
    id_ext_abb = ships_ext - ships_abb
    id_abb_ext = ships_abb - ships_ext
    id_abb_and_ext = ships_ext & ships_abb
    # get ships name
    name_ext_abb = [ships_name[i] for i in list(id_ext_abb)]
    name_abb_and_ext = [ships_name[i] for i in list(id_abb_and_ext)]
    # return
    return((set(name_ext_abb), set(name_abb_and_ext), set(id_abb_ext)))

mooc · python

Fri Oct 19 20:12:32 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-3-set.html

mooc - python - 3 - reference

by default all is reference !

==

Test equality of value

is

Test equality of reference (use copy.deepcopy() to secure it b = a[:] can create mistake with level of mutable)

COPY / DEEP COPY

Mutable elements propagate changes to shared references

a = [1, 2]
b = a
a[0] = 'shared'
print(b)

Use shallow copy for one level

b = a[:]
b = copy.copy(a)

Use deep copy for all levels

import copy
b = copy.deepcopy(a)

VARIABLE IDENTIFIER

https://docs.python.org/3/reference/datamodel.html#objects-values-and-types

id(v)

Return the identifier of variable

a = 3
b = 3
print(id(a), id(b))
b is a

CIRCULAR REFERENCE

http://pythontutor.com/

DELETE

https://docs.python.org/3/reference/simple_stmts.html#the-del-statement

del(v)

Delete variable

a = 10
del(a)
try:
    print('a=', a)
except NameError as e:
    print("a n'est pas définie")

delete a part

l = list(range(10))
del l[2:10:2]
l

delete few elements

d = dict(foo='bar', spam='eggs', a='b')
del(d['a'], d['spam'])

SHARED ASSIGNMENT

shared reference

b = a = []

cellule = [0]
liste = [cellule, cellule, cellule]
liste[0][0] = 1
print(liste, cellule)

shared assignment

liste = 3 * [[0]]
l[0][0] = 1
print(l)

copy assignment

liste = [[0], [0], [0]]
l[0][0] = 1
print(l)

assignment

+= *= ... # make default assignment by reference <=> make assignment by reference if variable is mutable

mutable

b = a = [1]
a += [1]
print(a); print(b); print(b is a); print(b == a)

not mutable

b = a = 1
a += 1
print(a); print(b); print(b is a); print(b == a)

mooc · python

Fri Oct 19 20:06:20 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-3-reference.html

mooc - python - 3 - path

PATHLIB

https://docs.python.org/3/library/pathlib.html

exists()

test if file exists

from pathlib import Path
filename = '/tmp/test'
path = Path(filename)
path.exists()
with open(filename, 'w', encoding='utf-8') as f:
    f.write('0123456789\n')
path.exists()

stat()

return statistiques about file

path.stat()
st_mode
st_ino
st_dev
st_nlink
st_uid
st_gid
st_size
st_atime
st_mtime
st_ctime

path.stat().st_size

from datetime import datetime
datetime.fromtimestamp(path.stat().st_mtime)
f"{mtime_datetime:%H:%M}"

unlink()

delete file

try:
    path.unlink()
except FileNotFoundError:
    print("no need to remove")

path = Path('/tmp')
for json in dirpath.glob("*.json"):
    print(json)

CLASS

path = Path('/tmp')
type(path)

from pathlib import PosixPath
issubclass(PosixPath, Path)

isinstance(path, Path)

FILE FORMAT

https://docs.python.org/3/library/fileformats.html

json

https://docs.python.org/3/library/json.html

limitations

Tuple, qui se fait encoder comme une liste / complex, set et frozenset, que l'on ne peut pas encoder du tout (sans étendre la bibliothèque)

pickle

https://docs.python.org/3/library/pickle.html

csv

https://docs.python.org/3/library/csv.html

SYS

import sys
autre_stdout = open('ma_sortie.txt', 'w', encoding='utf-8')
tmp = sys.stdout
print('sur le terminal')

sys.stdout = autre_stdout
print('dans le fichier')

sys.stdout = tmp
autre_stdout.close()

print('de nouveau sur le terminal')

with open("ma_sortie.txt", encoding='utf-8') as check:
    print(check.read())

OS

Old fashion

os.path.join()

ajoute '/' ou '' entre deux morceaux de chemin, selon l'OS

os.path.basename()

trouve le nom de fichier dans un chemin

os.path.dirname()

trouve le nom du directory dans un chemin

os.path.abspath()

calcule un chemin absolu, c'est-à-dire à partir de la racine du filesystem

os.path.exists()

pour savoir si un chemin existe ou pas (fichier ou répertoire)

os.path.isfile .isdir()

Pour savoir si un chemin est un fichier (et un répertoire)

os.path.getsize()

pour obtenir la taille du fichier

os.path.getatime()

et aussi getmtime et getctime pour obtenir les dates de création/modification d'un fichier

os.chdir()

change current directory

os.remove()

(ancien nom os.unlink), qui permet de supprimer un fichier

os.rmdir()

pour supprimer un répertoire (mais qui doit être vide)

os.removedirs()

pour supprimer tout un répertoire avec son contenu, récursivement si nécessaire

os.rename()

pour # renommer un fichier

glob.glob()

comme dans par exemple glob.glob("*.txt")

mooc · python

Fri Oct 19 20:06:00 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-3-path.html

mooc - python - 3 - file

https://docs.python.org/3/library/functions.html#open

options

r  # open for reading (default)
w  # open for writing, truncating the file first
x  # open for exclusive creation, failing if the file already exists
a  # open for writing, appending to the end of the file if it exists
b  # binary mode
t  # text mode (default)
+  # open a disk file for updating (reading and writing)
U  # universal newlines mode (deprecated)

f = open('file', 'options'[, encoding=])
f.close()

encoding

Necessary if you don't use option binary 'b'

f = open('/tmp/spam', 'w', encoding='utf8')
f.close()

f = open('/tmp/spam', 'bw')
f.close()

CONTEXT MANAGER

Close automaticaly file after treatment and exception

# writing
with open("foo.txt", "w", encoding='utf-8') as f:
    for i in range(2):
        f.write(f"{i}\n")

# reading
with open("foo.txt", "r", encoding='utf-8') as f:
    for line in f:
        print(line, end='')

iterator

with open("foo.txt", encoding='utf-8') as f:
    print(f.__iter__() is f)

ADVANCED

repr()

View real content

with open("foo.txt", encoding='utf-8') as f:
    for bloc in range(2):
        print(f"Bloc {bloc} >>{repr(f.read(4))}<<")

read()

Read all content of file

with open("foo.txt", encoding='utf-8') as entree:
    full_contents = entree.read()
    print(f"Contenu complet\n{full_contents}", end="")

flush()

Force writing to disk

binary mode

with open('strbytes', 'w', encoding='utf-8') as output:
    output.write("déjà l'été\n")
with open('strbytes', 'rb') as rawinput:
    octets = rawinput.read()
    print("on a lu un objet de type", type(octets))
    for i, octet in enumerate(octets):
        print(f"{i} → {repr(chr(octet))} [{hex(octet)}]")

Difference between characters & octets

with open('strbytes', encoding='utf-8') as textfile:
    print(f"en mode texte, {len(textfile.read())} caractères")
with open('strbytes', 'rb') as binfile:
    print(f"en mode binaire, {len(binfile.read())} octets")

mooc · python

Fri Oct 19 20:05:29 2018 * · permalink

http://code.ambau.ovh/snippets/mooc-python-3-file.html

PACKAGE

package states

package selection states

package flags

ACTIONS

OPTIONS

MAN

synopsis

options

Zero-address ``commands''

Zero- or One- address commands

Commands which accept address ranges

Addresses

base

Titre1

Titre2

Titre6

separator line

CONF

LIST

KEY

SERVER

FILE

TOC

CONFIG

COPY

DNSMASQ

LIST

IMAGE

INIT

NETWORK

PROFILE

STORAGE

EXEC

PUBLISH

TRICK

pretty print

json

yaml

profile

yaml

LXC

TOC

OPTIONS

Global options

Commit options

ADD

examples

DEL

FIX

UPDATE

UPGRADE

CACHE

examples

INFO

LIST

SEARCH

STATS

examples

DOT

POLICY

INDEX

FETCH

VERIFY

MANIFEST

RC-STATUS

RC-UPDATE

RC-SERVICE

FORMAT

examples

TOC

3 main modes

Normal mode

Command mode

Insert (edit) mode

NORMAL MODE

MOTIONS

arrow keys

word

line