...

Getting Started Check out the Guides section for: Quickstart Authentication Versioning Pagination Filtering External IDs API Guides See the following use case guides for best practices and sample code on common integrations: Telematics Routing Compliance & ELD Driver Documents Maintenance & ...

A place to share my love of tiny houses and small spaces... <br> <br> <script type="text/javascript"><!-- google_ad_client = "ca-pub-6821266315164954"; /* Google Recommended */ google_ad_slot = "1747729440"; google_ad_width = 180; google_ad_height = 150; //--> </script> <script...

Posts about github written by jason

I had long been aware of the [effort of cznic](https://gitlab.com/cznic/sqlite) to automatically translate the C source of SQLite to Go while avoiding the complexity CGo would introduce. SQLite was always meant to be a database you embed into your application, and I was curious what tradeoffs that meant, if any, and just generally how it differed from the usual suspects (PostgreSQL, etc.). To explore these trade-offs and differences, I decided to dive deeper into the usage of this library. SQLite is a single-writer multiple-reader database. Using just a single instance of database/sql.DB will not work well with this, because it will make it hard to avoid receiving SQLITE_BUSY errors when multiple writers try to run at the same time (because sql.DB is a connection pool). So we will need to separate readers from writers by making two instances of database/sql.DB and limiting the writer to a single [MaxOpenConn](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns) and the reader to as many as you require. So far, SQLite is behaving remarkably like other databases apart from the single-writer gotcha; we still have connections and per-connection settings. The usual way to pass these settings is usually in the DSN you pass to [sql.Open](https://pkg.go.dev/database/sql#Open) and this is [definitely possible](https://pkg.go.dev/modernc.org/sqlite#Driver.Open), but I find that it would be more straightforward if I could just run my custom SQL when a new connection is made[1]. We can do that with [sqlite.RegisterConnectionHook](https://pkg.go.dev/modernc.org/sqlite#RegisterConnectionHook). Now the question is what settings to use, because [there are quite a few of them](https://www.sqlite.org/pragma.html). These are the ones I am using personally: - `PRAGMA journal_mode = WAL;` Use write-ahead-logging. Better concurrency, but with an edge-case: with heavy writers, the wal can grow idefinitely ([see for more](https://www.sqlite.org/cgi/src/doc/wal2/doc/wal2.md)). I don't expect this will happen in my use-case. For a good description on the various jornal modes, [see this blogpost](https://www.mycelial.com/learn/sqlite-journal-mode-options). - `PRAGMA synchronous = NORMAL;` Since with the WAL journal mode, this is safe, there is no reason to use FULL. - `PRAGMA temp_store = MEMORY;` Let's store temporary indices, tables, etc. in memory, my use-case is not very resource-constrained to require using the filesystem (also, [this is a fun read](https://www.sqlite.org/fasterthanfs.html) if we are talking about additional syscalls to the kernel) - `PRAGMA mmap_size = 30000000000; -- 30GB` Just making sure, access to my database is always done with memory-mapped I/O. - `PRAGMA busy_timeout = 5000;` How much time to wait in milliseconds after which SQLITE_BUSY is returned when waiting for the db to unlock. - `PRAGMA automatic_index = true;` Make sure [automatic indexing](https://www.sqlite.org/optoverview.html#autoindex) is enabled, it can only help, and is a great way to get warnings about indexes that should be created. - `PRAGMA foreign_keys = ON;` Make sure foreign key enforcement is enabled. - `PRAGMA analysis_limit = 1000;` Make sure [ANALYZE](https://www.sqlite.org/lang_analyze.html#approx) returns quickly in case the DB gets large. - `PRAGMA trusted_schema = OFF;` No idea what security audited functions are, but this setting makes sure only those can be run. The manual page tells me to turn it off anyway, so let's do that. Not all of these settings are per-connection settings, but it doesn't hurt to always run all of these on every connection, and having a single "Init SQL" makes things simpler. Let's deal with explicit transactions next. The default transaction mode is DEFERRED, meaning that it depends on the next statement what type of transaction is started (if a SELECT statement comes first, a read transaction is started, with the possibility of upgrading it to a write transaction depending on the statements that follow). With our setup, there should never be multiple writers concurrently, so making all transactions write transactions from the get-go (with BEGIN IMMEDIATE) can only help us show if this invariant is violated. Putting this all together, this is the code we arrive at: ```go package main import ( "bytes" "context" "crypto/sha1" "database/sql" "database/sql/driver" "encoding/hex" "fmt" "time" "modernc.org/sqlite" ) type DB struct { dbPath string Read *sql.DB Write *sql.DB } const initSQL = ` PRAGMA journal_mode = WAL; PRAGMA synchronous = NORMAL; PRAGMA temp_store = MEMORY; PRAGMA mmap_size = 30000000000; -- 30GB PRAGMA busy_timeout = 5000; PRAGMA automatic_index = true; PRAGMA foreign_keys = ON; PRAGMA analysis_limit = 1000; PRAGMA trusted_schema = OFF; ` func New(ctx context.Context, dbPath string) (db *DB, err error) { db = &DB{ dbPath: dbPath, } // make sure every opened connection has the settings we expect sqlite.RegisterConnectionHook(func(conn sqlite.ExecQuerierContext, _ string) error { _, err = conn.ExecContext(ctx, initSQL, nil) return err }) write, err := sql.Open("sqlite", "file:"+db.dbPath) if err != nil { return } // only a single writer ever, no concurrency write.SetMaxOpenConns(1) write.SetConnMaxIdleTime(time.Minute) if err != nil { return } read, err := sql.Open("sqlite", "file:"+db.dbPath) if err != nil { return } // readers can be concurrent read.SetMaxOpenConns(100) read.SetConnMaxIdleTime(time.Minute) db.Read = read db.Write = write return } func (db *DB) InTransaction(ctx context.Context, tx func(context.Context, *sql.Conn) error) (err error) { // cancel anything that takes longer than 5 seconds var cancel func() ctx, cancel = context.WithTimeout(ctx, time.Duration(5*time.Second)) defer cancel() // separate conn thats just for us // so that the transactions queries are executed together conn, err := db.Write.Conn(ctx) if err != nil { return } defer conn.Close() _, err = conn.ExecContext(ctx, "BEGIN IMMEDIATE TRANSACTION") if err != nil { return } defer func() { if r := recover(); r != nil || err != nil { if _, err = conn.ExecContext(ctx, "ROLLBACK"); err != nil { return } } }() err = tx(ctx, conn) if err != nil { return } _, err = conn.ExecContext(ctx, "COMMIT") return } ``` Just as a small added bonus, it's also very easy to register our own go functions that we can then call from SQL. In this example, we define the sha1 function, receiving and returning a single argument, and always returning the same output for the same input (thus the deterministic). For more information, [see the docs](https://www.sqlite.org/appfunc.html) ```go func registerFuncs() { // lets add our own sha1 hash function usable from SQL sqlite.MustRegisterDeterministicScalarFunction( "sha1", 1, func(ctx *sqlite.FunctionContext, args []driver.Value) (driver.Value, error) { var arg *bytes.Buffer switch argTyped := args[0].(type) { case string: arg = bytes.NewBuffer([]byte(argTyped)) case []byte: arg = bytes.NewBuffer(argTyped) default: return nil, fmt.Errorf("expected argument to be a string, got: %T", argTyped) } h := sha1.Sum(arg.Bytes()) return hex.EncodeToString(h), nil }, ) } ``` In conclusion, this solution works perfectly well for most of the usage scenarios out there where immense scaling is neither expected nor needed. There are faster ways to use SQLite with Go (see the [excellent cvilsmeier benchmarks](https://github.com/cvilsmeier/go-sqlite-bench)), but its still plenty-fast enough, and avoiding CGo makes it very convenient also. [1]: This is another thing that database/sql.DB being a connection pool makes harder.

⬛️ CLI tool and library for saving complete web pages as a single HTML file - Y2Z/monolith

FreePBX 17 Installation Script. Contribute to FreePBX/sng_freepbx_debian_install development by creating an account on GitHub.

Auf YouTube findest du die angesagtesten Videos und Tracks. Außerdem kannst du eigene Inhalte hochladen und mit Freunden oder gleich der ganzen Welt teilen.

Auf YouTube findest du die angesagtesten Videos und Tracks. Außerdem kannst du eigene Inhalte hochladen und mit Freunden oder gleich der ganzen Welt teilen.

United States or Remote

A script that you can run in the background! Contribute to 21y4d/nmapAutomator development by creating an account on GitHub.

First, I want to say that there is so much information to go through on earthships that you really should check out the website: http://earthship.com/.  I will try to give you a simple explanation, but I'm sure I will leave things out.  Earthships are homes that are entirely self sufficient.  They grow food, run off solar…

Auf YouTube findest du die angesagtesten Videos und Tracks. Außerdem kannst du eigene Inhalte hochladen und mit Freunden oder gleich der ganzen Welt teilen.

Netbox plugin to sync data between LibreNMS and Netbox - bonzo81/netbox-librenms-plugin

A clone of multimon-ng supporting SDL Audio Drivers by Chris Zenzel - czenzel/multimon-ng

MikroTik Wireless systems, Switches, Ethernet routers, RouterBOARD products, Antennas and Accessories

Idealist connects millions of idealists – people who want to do good – with opportunities for action and collaboration all over the world.

Open Source Security Compliance Solution. OpenSCAP has 19 repositories available. Follow their code on GitHub.

Hacker Public Radio is a podcast that releases shows every weekday Monday through Friday. Our shows are produced by the community (you) and can be on any topic that is of interest to hackers and hobbyists.

Auf YouTube findest du die angesagtesten Videos und Tracks. Außerdem kannst du eigene Inhalte hochladen und mit Freunden oder gleich der ganzen Welt teilen.

Configuring the WiFi is probably the first thing you'll do on your Raspberry Pi. Here are three methods to meet all your WiFi configuration needs!

  Ah the 1970s, when disco dust was plentiful and there were cocaine paraphernalia ads galore in head magazines. Dig the Hoover-themed coke spoons! Or the “what the hell were they thinking” handmade ivory straws. And if your nose is a little clogged from too much coke, why not try “Noze: the nose wash”? So as the majority of the taglines in these magazine clippings say, “Let it snow!”                                           via The World’s Best Ever