Engineering notes since 2020
Hands-on articles about Architecture, IT infrastructure, DevOps, Security, and AI/ML — tested on real hardware, written down so they can be reused.
AI assistants now write a large share of the code in many teams. Yet delivery often doesn’t get faster. In Google’s DORA 2024 research, as AI adoption grew, individual productivity went up, while software delivery stability fell by an estimated 7.2% and throughput by 1.5%.
I thought that the best option to run Frigate is to run bare metal and skip virtualization and system containers. However now situation changed a little bit as I was able to fire up Frigate on LXC container on Proxmox with little help of AMD ROCm hardware assisted video decryption.
We are using Huawei PV installation on top of our house roof. It has web panel and application available to preview every detail about its settings and working conditions. However I would like to integrate PV power production into my Fibaro HC3.
During sync between two Proxmox Backup Server instances I got “decryption failed or bad record mac” error message. So I decided to go for upgrading source PBS to match its version with target PBS. PBS upgrade To upgrade PBS: Get rid of bookworm sources.
Add “alias” to /etc/network/interfaces: And restart network interfaces.
Imaging you are running TrueNAS virtualized and would like to resize drive increasing its capacity. Then reboot and new size should be visible in ZFS pool. It has ability to auto expand by default.
In addition to my previous article about GitLab service desk feature with iRedMail I would like to enhance it a little bit my fixing some missing parts. Starting with connecting to PostgreSQL vmail database: Verifying mail server connectivity and TLS issues. GitLab (and iRedMail also) does not allow expired certificates.
Failing Proxmox node and unable to access /etc/pve means that you have broken pmxcf.
I got 2.5GbE PCI-e card, but no drivers present in pfSense installation. To list devices: There is this Realtek network card: But there is no driver enabled: I installed realtek-re-kmod-198.00_3 package.
Last time (somewhere around 2023) there was an option on Scaleway to install Proxmox 7 directly from appliance. There was also possiblity to use Debian 11 and install Proxmox atop of it.
Instead of using dedicated Hetzner servers you can use their cloud solutions for less demanding use case scenarios. The idea is to have one server for pfSense and the other one for regular private use, hidden behind gateway.
So we decided to try Zigbee wall switches and many of available brands work with Fibaro. But not all of them and not in all configurations. Even if some switch or module works it may only work with Tuya gateway and application and not with Fibaro gateway.
I use NextCloud as alternative to Google Photos. My smartphone sends pictures over OpenVPN to my NextCloud instance. However for over a year Android application has been showing notification that server version is outdated, and from time to time automatic upload do not work.
At Indatify we use PostgreSQL for most of the time when dealing with data. We tried to use Cassandra, but for now it is too much of a constraint in such dynamic data environment. We put it on shelf.
In our lab we have 2 x HP z800 workstations. It is somehow ridiculous piece of hardware in terms of today standards. It’s got over 1 kW power supply, dual CPU motherboard and 12 memory slots.
Managing self-hosted Sentry is pretty straightforward. However due to high number of containers which Sentry is comprised of, there is high chance that we can encounter some kind of issues due to obvious nature of software development products which consists by definition some flaws.
As for year 2025 there is no more updates for CentOS 7.9.2009. Moreover you need to adjust repository file as network locations changed a little bit. So edit /etc/yum.repos.d/CentOS-Base.repo as follows: Then recreate yum cache: And if necessary: Now you are good to go with packages installation.
I thought of adding another Proxmox node to the cluster. Instead of having PBS on separate physical box I wanted to have it virtualized, same as on any other environment I setup. So I installed fresh copy of the same Proxmox VE version and tried to join the cluster.
Lets say you want to pass traffic from your local container/VM via some external pfSense box. This way there is no need to setup VPN on each container you want to include in the setup.
IP2Location is a IP address databases where you can find the latest IP to location associations. The complete IPv4 range is 4 294 967 296 addresses which is 32 bit. IP2Location contains 4 291 944 710 addresses which is a little less.
From 8k to 29k writes per second We took IP2Location version DB11 database. It holds few millions of IPv4 ranges which should unwrap onto over 2 billion addresses. Such number of entries is actually not a big deal for PostgreSQL RDBMS or Apache Cassandra distributed databases system.
Recently my pfSense running on the same hardware for almost 3 years, died. I tried rebooting it and removing RAM, cards etc, with no luck. So decided to bring it back from configuration backup onto new drive.
Distributed, partitioned, multi-master, increment horizontal scale-out NoSQL data manegement system for global mission-critical use cases handling petabyte sized datasets Dynamo, Amazon, Facebook, Apache… “Reliability at massive scale is one of the biggest challenges we face at Amazon“ source: “Dynamo: amazon’s highly available key-value store” In 2004 there have been…
Have you ever wondered how many tables can we create and use in PostgreSQL database server? Shall we call them partitions or shards? Why not to use built-in “automatic” partitioning? Partitions or shards? Lets first define the difference between partitions and shards.
Even wondered how computer is built? And no, I’m not talking about unscrewing your laptop… but exactly how the things happen inside the CPU. If so, then check out TINA from Texas Instruments and open my custom-made all-in-one computer. I spend few weeks preparing this schematic.
Imagine you would like run 2 x vCPU and 4 GB RAM virtual machine. Which service provider do you choose? Azure, AWS or Hetzner? With AWS you pay 65 USD (c5.large instance… and by the way why this is called large at all?).
Ever wondered what can be the highest load average on the unix-like system? Do we even know what this parameter tells about? It shows the average number of either actively running or waiting processes.
I defined Docker Swarm cluster with 20 nodes and created service using Nginx HTTP server Docker image. I scaled it to 1000 container instances, which took a while on my demo hardware.
If you wonder how to automatically deploy 20 nodes of Docker Swarm and run 100 Docker containers in it, then continue reading. I will show how to achieve this by using Terraform, Ansible and Portainer.
If you wonder how to automate Ubuntu virtual machine creation and then deploy it to Proxmox in multple copies, then you are looking for Packer and Terraform. Side note: going for virtual machines in Proxmox is the proper way.
The article explores the use of ArchiMate, a modeling language, for enterprise architecture, focusing on its application through the Archi tool. ArchiMate offers a structured approach to modeling complex IT systems by providing various layers such as motivation, strategy, business, and technology.
Create secure, high-performance, affordable environment for your container applications using Hetzner dedicated servers.For around 50€ per month. This setup can also be done using different server providers, both dedicated and shared, even on public cloud. This tutorial has not been sponsored by Hetzner or any other software vendors.
If you have Sentry on-premise installed and run out of space, then bringing it back can be a little bit tricky. Especially if you have tons of projects configured with loads of data coming into it.
In order to process incoming e-mail messages as issues registered in GitLab you need to have catch-all mailbox and GitLab configured to handle it. However, there are few quirks you need to know when setting it up. Assuming you have GitLab installed as well as iRedMail.
Building applications to do things better leads to operational excellence. And using digital tools and applications across the enterprise to do business differently – to enable innovation and creativity – leads to strategic advantage.
In order to separate Proxmox node from cluster:
If you would like to run virtual machine inside another virtual machine, then you need to have CPU with nested virtualization feature and this feature needs to be enabled.
Installing Redash is straightforward if you use https://github.com/getredash/setup repository. Just run setup script to download, install and configure all required packages. But there is one thing missing from the default. It is mail configuration.
Benefit from running your own docker registry is cost and privacy. But in order to use it in various places you need to secure its connection with SSL/TLS certificates.
In case you have some LXC containers on your Proxmox server, then there is high chance that you will get some errors during backup them up. Some container templates may not support snapshot or suspend modes. Instead you should you use stop mode.
Bare metal servers offered at Scaleway are at reasonable prices, which can be compared to those on Hetzner. You even get Proxmox installation by default contrary to Hetzner were you need to install Debian first.
If you wonder if it is possible to have private-only Proxmox cluster in Hetnzer, then the answer is yes, almost. Of couse you can order dedicated hardware to hide your boxes from public eyes, but if you are not going that way, then you can try other way.
It might sound a little weird, but that’s the case. I was trying to setup NFS mount in OKD docker registry (from this tutorial). During oc rsync from inside docker-registry container I found that OKD master processes are down because of heath check thinking that there is some connectivity problem.
Logging all HTTP traffic is often unnecessary. It especially applies to website which include not only text content but also all kind of additional components, like JavaScripts, stylesheets, images, fonts etc.
First of all this SSD drive which I use is somehow faulty. It is a Goodram SSDPR-CX400-01T-G2 drive of 1TB. It have been working fine for few weeks until some construction worker made a electric short causing some abnormal frequences in wires resulting a faulty drives and memory sticks.
Having your own mail server could be useful but also sometimes dangereous. I am happy to see appliance such as iRedMail which cover variaty of topics regarding a somehow complete solution. I pick Ubuntu 22 on Hetzner.
You can use OpenStreetMaps on your own hardware. You need to grab map files first, which can be found at https://download.geofabrik.de.
Data is valuable if consumed or at least identified. For private and corporate usages I suggest installing Redash as it gives options for saving queries, exporting data, creating visualizations and dashboards and also setting up alerts.
As already stated in my year 2023 plans, there will be some time for graphics programming. Back in the days I was exploring OpenGL but that was way long ago. Nowadays I think to try something much simpler and accessible like for instance WebGL.
I’m looking forward for new year’s technology opportunities. I have few ongoing projects which I would like to finish by the end of this year.
I thought that installing pg gem on my clean Ubuntu 22 will be easy, but no. I got some weird message: So I tried to force installation: After this: On this Ubuntu 22 release installing Ruby interpreter from packages you got 3.0.2p107.
SMB shares can be created either with UI on a desktop version or command line on a server installation. You can use this shares both on Linux stations and Windows boxes as well. We are going to create editors group and add current user to this group.
In case you use ever changing outbound public IP connection like in Microsoft Azure, then you can try create machine with public IP and passing your local traffic to remote site via simplaproxy. L switch is for local and R is for remote.
Finally after OKD 3.11 support has ended I’ve decided to try 4.x releases. I found that there is quite nice installation assistant available on console.redhat.com (Red Hat Hybrid Cloud Console). So I tried it and installed new cluster on my dedicated hardware.
I installed Proxmox Backup Server several times, both on bare metal and virtualized inside Proxmox VE. This time I did it in PVE as virtual machine and encountered an issue while initializing disk with GPT, creating directory, LVM or ZFS.
There is not too much a precise documentation from Hetzner available if you want to know what exactly you should do to run dedicated servers with primary and secondary public IP, virtual machines and vSwitch. There are some articles but they are written in non-informative way.
If you use OKD/OpenShift then most probably you also run internal and private Docker registry for your builds. Cluster uses this to lookup for containers images for further deployment. For basic, default installation your Docker Registry is located in a project called default.
Since the beginning of 3.x line of OpenShift/OKD releases there are various issues with internal certificates. TLS communication inside the cluster is used in several places like router, registry, compute nodes, master nodes, etcd and so on.
In case you have been low on disk space on your Elasticsearch instance, there is high probability that your indices are marked read only now. In order to fix this one, first either delete/archive indices or increase your disk space.
In one of my previous posts I mentioned some troubles regarding reinstalling Ubuntu 22, loosing ability to select OS and to boot at all actually. I found that Ubuntu 20 recognizes properly my fresh Windows installation but Ubuntu 22 does not.
There are two ways of expanding your root filesystem space. It’s either by adding additional volumes or by resizing PV. Let’s try the latter. We have CentOS 7 wth XFS running on Proxmox. First expand drive size with admin UI.
I have dual boot on my Lenovo Thinkpad T420s, Windows 10 and Ubuntu 22. Actually I had, because I tried to reinstall Ubuntu 22 and I’ve lost my dual boot and ability to boot at all.
I was replacing drives and memory in one of servers and out of a sudden node was unable to start. When finally it booted it was gone out of cluster. Weird. This node has been installed on SSD drive which had 2 years or constant runtime.
During installation Ubuntu will create LVM with half of the space available. In order to expand it to whole space available you need to extend logical volume and expand file system as…
Display geo location map for NGINX traffic logs in Kibana Summary There are 3 things to remember and configure in order to have geo location map working: Use “forwardfor” option on pfSense HAProxy TLS frontend Enable filebeat NGINX module and point particular log files Define custom NGINX log format…
Among 3 million public IP address ranges for the whole world, 68k belongs to the Russian Federation. This translates into 45 million addresses. Scanning the HTTP port on this population took 20 hours. I obtained 630k IP addresses with listening on port 80.
The basis for the construction of diodes and transistors A semiconductor is a material whose electrical conductivity is between the conductors and insulators. Their resistance and conductivity depends on temperature and admixtures. [W] The most commercially used semiconductors are those based on silicon, and in the past also on germanium.
Energy in a magnetic field [209] Just as a capacitor works by storing energy as an electric field, an inductor stores energy as a magnetic field. The energy stored in the coil is calculated in terms of current.
I will start with the werid experience with one of WP themes – Polite and Polite Grid. I was wondering why my website make double requests on every page. One for the document and other for content. This was annoying as I was unable to measure traffic properly.
Lowering the voltage value [202] The next example I want to present in practice is a popular voltage divider. It allows the output voltage to be reduced to the desired value due to the properties described a little earlier.
Theory and practice [W] One of the basic principles of current flow is the first law of Kirchhoff, which says that for the electric circuit node, the algebraic sum of flow rates is zero.
Signal filtering [202] Capacitors are divided into polarized and non-polarized depending on whether the direction of their inclusion in the circuit is important. Generally speaking, capacitors filter the waveform of the power source.
In case you’ve managed to overcome other issues with system unable to boot, disk size not updated then there might be still one problem left. Sometimes resizing with gpart works fine but file system resize does not.
Following previous struggles on pfSense FreeBSD on Azure. After resizing disk Microsoft starts at some point adding an another drive which is temporary one. I do not need this by any means.
Recently I described how to increase drive, partition and filesystem on FreeBSD in Azure virtual machine. It turned out that this scenario was little bit different from a virtual machine running on top of VMware.
After 2 years on DigitalOcean I’ve decided to move out. Not because it’s bad or expensive, but because I need one more feature which is IDS/IPS. By default WordPress installation on DO contains fail2ban, which is fine, but I find Suricata way more powerful.
If you log in to Proxmox’s shell and see error stating that “setting locale failed“, then execute the following command to get rid of that message: Not quite sure why it happens, but I see it almost all the time.
I use NetGate pfSense+ version 22 based on FreeBSD 21. I encountered a problem of running out of disk space because of packet logging enable in Suricata. Out of a sudden there was high traffic and therefore high logs production. Disk space utilization was over 100%.
Starting from PostgreSQL 10 we have available new type of paritioning, which is declarative one. Before we had inheritance which is also good but has some limitations. Now, with declarative partitioning user need not to specify partition name while inserting data.
On many chances you may need to configure some domains using .local suffix. Unfortunately this is covered by RFC 6762 (multicast DNS).
Image a hypothetical scenario having two 512 GB drives and want to use a Proxmox ZFS VM replication onto a second server with one 1 TB drive. Solution is quite simple.
Having LVM on the disk causes system to automatically active such volumes. I installed used drive with Ubuntu on it to my Proxmox server. Using Proxmox’s UI you will not be able to wipe this drive, because volume group has been already auto-activated.
Aborting Ubuntu packages update via Ansible gave me some weird state of libatk-wrapper-java-jni package. My playbook installs default-jre and it was painfully slow so I thought that there is some lockup, but there was not.
Changing Java language level to run compiled code on older runtimes I’m working on some Java project. I use IntelliJ IDEA and deciced to go with Oracle’s OpenJDK-18. But… this runtime is available by default only during compilation in the IDE. In the system I have OpenJDK-11.
The most recent min.io server release requires one additional thing in the configuration comparing to versions in the past years. Having min.io on one box and NGINX on another one requires setting up a reverse proxy, which is straightforward operation.
In case you would like to enable some PostgreSQL extensions there is basically one thing you should do which is install postgresql14-contrib package and then fire CREATE EXTENSION from PostgreSQL itself.
I own a preowned HP z800 workstation. As purchased it contained HDD inside – Western Digital WD5000AAKX-75U6AA0, SATA III, 500GB, 7200 RPM, 16MB of cache. Unfortunately it passes SMART and do not show wearout metric.
Docker Registry could be useful in various scenarios, but the most common would be containers orchestration software such as Kubernetes, OpenShift/OKD or Nomad.
I was trying to add some new packages and figured it out that my Ubuntu 21 is out of date. Unfortunately some time ago I unchecked all repositories in Software & Updates so my /etc/apt/sources.list was empty.
Latest Clustercontrol – version 1.9.4 – installation somehow requires pg_stat_statements to be activated in PostgreSQL 14. I have never noticed it before, both on 9.6, 12 and 14. This is simple fix.
If you need to mirror remote repository and push it to another remote repo, then there is one thing you should do to be able to export all remote branches and not only the master.
Using CentOS7 templates on Azure could result in lack of LVM, swap and non auto-extendable partitions. Some other templates have such feature, but not all of them. In case you assigned bigger disk in Azure portal you need still to do few things.
Old Zabbix Proxy 4.2 installation could be quite confusing as you cannot grab the latest one to work with old Zabbix server. Fortunately there is archive packages repository at repo.zabbix.com/zabbix/4.2/ubuntu/pool/main/z/zabbix/. You need to keep track which operating systemu should be used.
If you happen to use Proxmox’s LXC which stands for Linux Containers, then you might be interested in this one. Proxmox has this turnkey containers available to download from its UI. Among them there is WordPress.
PostgreSQL 14 has few improvements over previous versions, especially older ones, like 9.6. This includes automatic data direction using partitioning. So, let’s install it on CentOS 7. Then you should login to the database and set password: Basic configuration includes firewall setup (if it’s active), access control and performance settings:
Theory and practice [204] This is the SI unit of resistance. 1 ohm is the electrical resistance between two points on a conductor when a constant potential difference of one volt between the two points causes a current of one ampere in the conductor.
Construction and testing of circuits in the TINA TI program To translate theory into practice, we can use the TINA package from DesignSoft for Texas Instruments. It is a tool for building virtual analog and digital circuits.
Measure of the strength of electric charges. Electrostatic potential difference. Electronics is based on physics. The basic concepts here are voltage, current and resistance. We should start with that. The voltage is therefore a measure of the strength of electric charges.
In case you encounted power outage or just blindly pressed restart button on your machine, there is high chance you will have some issues with your cluster. Mine has only one node, so there is no redundancy. If I lose indices then data is gone.
I already have over 10 virtual machines in my home virtual environment. It is the number I find enough to start using some automation tools like Ansible. Installing Ansible on Ubuntu or CentOS systems is straightforward, just use regular apt or yum commands.
If you happen to have some Bash scripts and would like to abort it in clean way there is way to do it. You need to trap ctrl+c command. In case you have some while loop in your scripts, without it, pressing any abort commands is only half way effective.
Once you have installed Elasticsearch and Kibana servers you need to grab some data. First choice could be beats modules, either Metricbeat or Packetbeat. There is also Filebeat.
Recently I’ve been tryout Elasticsearch 7 with Kibana and Metricbeat as well as Filebeat. I encountered some issues with indices so I dropped dashboards and visualizations and tried to load them again. I also cleared indices and patterns thru index management page.
Get rid of ads and stop sending your data for free to Opera Little history My favourite browser was Opera for so many years. Between 2000 and 2005 it was adware showing, well… ads. In 2005 ads have been remove as the financing came from Google, Opera’s default search engine.
If you run digital services platform or critical infrastructure then most probably you are covered by NIS 2 and its requirements including those concerning information security.
Keep your data safe. Device loss or unauthorized access can be mitigated by encrypting drive in your server and workstation. So you may have a Ubuntu Linux installation on your bare metal or virtual machine. Does it have an encrypted drive?
Backup is a must. Primary hardware fails. Local backups can also fail or can be inaccessible.
Due to previous failure of SSD drive from Goodram I was forced to use brand new 1TB HDD from Toshiba. It was not a problem because the system running on it mainly have been using writes with not too much reads.
In case of an Abuse message from BSI (in Germany), the following services must be enabled and disabled: Verify in 2 ways: Cutting with DROP traffic by Suricata IPS is not sufficient, because not all possible traffic signatures on port 111 are used and in most cases this port…
Once in a while there is outage in my main internet connection. In order to keep everything up and running (fortunately) we can setup secondary WAN over LTE connection. Start with connecting your device (e.g. LTE router) over ethernet cable to pfSense box.
For non-users of either Proxmox or Suricata: the first one is virtualization appliance which helps firing up virtual machines as well as LXC containers and the latter is network traffic security system which is able to identify (IDS mode) or even block malicious traffic (IPS mode).
Until recently I though that having DNS subdomain entries provides enough obscurity thus should it be secure. If your DNS server does not offer transfering domain to another place then any subdomains should be hidden from public sight.
In case you have access to MikroTik device without UI available, only CLI then you possibly would like to know how to do some basic reconfiguration to gain access to admin panel listening on local network. So… to change user password: To see all the configuration: To create PPTP VPN:
In case you would like to have PPTP VPN server but no such is installed. But first things first. For such an obsolete hardware and software keep in mind that accessing it requires passing old host key algorithms and cipers.
Until recently I did not know that honeypots I encounter are based on Cowrie. But it does not change much to be honest. I already seen almost 100 of these while randomly exploring things on the internet. It was first thought that they are honeypots, by pure intuition.
If for some unknown reasons you need to connect thru SSH to some outdated server using Ruby code, then you are in the right place. But first a little piece of theory behind it. There are 4 configurable things in OpenSSH concerning security configuration: KexAlgorithms: per-connection keys (key exchange) HostkeyAlgorithms:
If you own this router then you might be interested how to setup VPN Server. I am personally interested in a way of how to do this using CLI only. It looks easy however there is one odd settings you should be aware of.
On Wive-NG router when talking about VPN you have either PPP or WireGuard. As PPP is first listed so I tried it before trying WireGuard. Here are my thoughts. Configuration is pretty simple. In the router menu, go to Services – VPN Server. Then select PPTP protocol.
Possibility that you own or operate EcoRouter 2008 with EcoBNGOS are very low. However just in case you will have to work with this one it will be useful to know at least some basic commands.
Want to change default DNS for the network in MikroTik’s RouterOS CLI? You can use web UI but it could have some issues preventing from configuration save, so it is nice to know how to use it via CLI.
Disconnecting wireless clients from access points The problem The nature of wireless networking is quite problematic because transmission goes over air and can be intercepted by anyone. Of course there is data packet stream encryption. But deauthentication frames are not encrypted and can be forged.
Detecting and preventing threats Introduction To provide security in a network you can deploy IDS or IPS systems. The difference is on the second letter, D stands for detection and P for prevention.
I want to enable only particular rules categories. Do not want to have all these internal Suricate rules as they cover too broad variaty of cases including loads of false-positive.
Programowanie CPU i GPU “Mamy rok 2022. Minęła ponad dekada od pojawienia się OpenCL. Omówię standard, przykładowy sprzęt, a także zaprezentuję rzeczywiste przykłady, które można wykorzystywać obecnie zarówno na historycznym jak i najnowszym sprzęcie, co ma pokazać swego rodzaju uniwersalność tego rozwiązania. Przy okazji zobaczymy jak ewoluowała technologia na przestrzeni ostatnich lat i czy teoria odpowiada praktyce.
Podstawy elektroniki i budowa komputera w symulatorze “Jest to pierwsza część serii Simple High Performance Computing o tytule Podstawy elektroniki i budowa komputera w symulatorze. Swoim zakresem obejmuje zagadnienia związane z elektroniką, a konkretnie jej podstawami, począwszy od pierwotnych zasad fizyki przekładających się na komponenty elektroniczne, z których można budować złożone układy, a finalnie najprostszą obliczeniową maszynę cyfrową.
“Był początek lat dziewięćdziesiątych, kiedy przed moim mężem pojawiła się szansa otrzymania działki pracowniczej. Co prawda na razie było to wysypisko śmieci na obrzeżach istniejącego kilkanaście lat Ogrodu. Czekała nas przeogromna praca. Nasze dzieci mówiły na ten kawałek ziemi „Kambodża”. Musieliśmy wywieźć sterty śmieci, złomu i szkła…” – Ridero. 2019 (ISBN 9788381893862).
“Dzięki książce “Ruby on Rails. Ćwiczenia” opanujesz podstawy tworzenia aplikacji internetowych za pomocą tej technologii. Nauczysz się programować w języku Ruby: poznasz jego elementy i zasady projektowania obiektowego, metody przetwarzania danych tekstowych, pracy z plikami i katalogami oraz obsługi błędów i wyjątków.
“Czym jest jakość oprogramowania? Czy to wyłącznie miara technicznej doskonałości kodu? A może jakość to coś znacznie więcej? Czy niezawodne działanie i brak błędów świadczą o wysokiej jakości programu? Co charakteryzuje kod, który można uznać za dobry? Jak mierzy się jakość w projektach informatycznych?
“Ruby to nowoczesny, wieloparadygmatowy, interpretowany język programowania. Wraz z platformą Rails stanowi jedno z najpopularniejszych rozwiązań służących do szybkiego tworzenia aplikacji sieciowych; wspiera wiele znanych serwisów dostępnych w internecie.
Quick overview of LLM MLX LORA training parameters. weight_decay A regularization technique that adds a small penalty to the weights during training to prevent them from growing too large, helping to reduce overfitting.
Full fine-tuning of mlx-community/Qwen2.5-3B-Instruct-bf16 Recently I posted article on how to train LORA MLX LLM here. Then I asked myself how can I export or convert such MLX model into HF or GGUF format.
I have done over 500 training sessions using Qwen2.5, Qwen3, Gemma and plenty other LLM publicly available to inject domain specific knowledge into the model’s low rank adapters (LORA). However, instead of giving you tons of unimportant facts I will just stick to the most important things.
YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and industrial communities. For more details, please refer to our report on Arxiv. https://yolox.readthedocs.io/en/latest/demo/onnx_readme.html https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/ONNXRuntime To configure this in Frigate:
Upload image, enter text prompt and press Start Generation. It is as easy as it looks like. So we take some pre-trained models, feed it with some text prompt and starting image and things happen on GPU side to generate frame by frame and merge it into motion picture.
In previous article about GPU pass-thru which can found here, I described how to setup things mostly from Proxmox perspective. However from VM perspective I would like to make a little follow-up, just to make things clear about it.
How about AI chatbot integraton in you Mattermost server? With possiblity to generate images using StableDiffusion… So, here is my Indatify’s Mattermost server which I have been playing around for last few nights.
I thought that installing NVIDIA RTX A6000 ADA in default Ubuntu 22 server installation would be an easy one. However, installing drivers from the repository made no good. I verified if secure boot is enable and no it was disabled.
LLMs contain built-in policies for protecting minors, animals etc. Monkey eating sausage should be against policy. But it can be fooled and finally model stops complaining and describe what we want to. Tried: to generate funny/controversial pictures.
Generate WebGL, Three.JS, HTML, CSS, JavaScript, no Python code, single page with rotating cube, ambient lighting. Load libraries from CDN. Let ambient lighting be as such cube edges are visible. add directional lighting also pointing at the cube. Scene needs to be navigable using arrow keys. Ensure browser compability.
My local toolbox was empty, now it’s full. Lately I have been writing about Ollama, WebUI and StableDiffusion on top of Automatic1111. I found myself struggling a little bit to keep up with all those information about how to run it in specific environments.
I want to create custom chatbot experience. I want to be based on Google’s Gemma AI Large Language Models. I find Gemma3, especially 27b version very capable while problem solving. It has been trained on such data that I find it interesting.
slight utlization drop when dealing with multi GPU setup TLDR Power usage and GPU utilization varies between single GPU models and multi GPU models. Deal with it.
With Ollama paired with Gemma3 model, Open WebUI with RAG and search capabilities and finally Automatic1111 running Stable Diffusion you can have quite complete set of AI features at home in a price of 2 consumer grade GPUs and some home electricity.
Utilize both CPU, RAM and GPU computational resources With Ollama you can use not only GPU but also CPU with regular RAM go run LLM models, like DeepSeek-R1:70b. Of course you need to have fast both CPU and RAM and have plenty of it.
Ollama with WebUI on 2 “powerful” GPUs feels like commercial GPTs online I thought that Exo would do the job and utilize both of my Lab servers. Unfortunately, it does not work on Linux/NVIDIA with my setup and following official documentation.
Theory: running AI workload spreaded across various devices using pipeline parallel inference In theory Exo provides a way to run memory heavy AI/LLM models workload onto many different devices spreading memory and computations across.
No, cant use Tesla K20xm with 6GB VRAM for modern computation as it has Compute Capability parameter lower than required 7.0. Here you have table of my findings about libraries/frameworks, required hardware and its purpose.
What is Qwen? This is the organization of Qwen, which refers to the large language model family built by Alibaba Cloud. In this organization, we continuously release large language models (LLM), large multimodal models (LMM), and other AGI-related projects. Check them out and enjoy! What models do they provide?
Well, in one of the previous articles I described how to invoke Ollama/moondream:1.8b using cURL, however I forgot to tell how to even run it in Docker container. So here you go: You can specify to run particular model in background (-d) or in foreground (without parameter -d).
What is OpenVINO? “Open-source software toolkit for optimizing and deploying deep learning models.” It is developed by Intel since 2018. It supports LLM, computer vision and generative AI. It runs on Windows, Linux and MacOS. As for Ubuntu, it is recommended to run on 22.04 LTS and higher.
Given this image: You would like to describe it using Ollama and moondream:1.8b model you can try cURL. First encode image in base64: Then prepare request: And finally invoke cURL pointing at your Ollama server running: In response you could get somehing like this:
These are two majors which allow to run object detection models. Google Coral TPU is a physical module which can be in a form of USB stick. TensorRT is a feature of GPU runtime. Both allows to run detection models on them.
For those using ZoneMinder and trying to figure out how to detect objects, there is deepquestai/deepstack AI model and builtin HTTP server. You can grab video frames by using ZoneMinder API or UI API: You need to specify address, monitor ID, user, password.
Did you know that you can use the Polish LLM Bielik from SpeakLeash locally, on your private computer? The easiest way to do this is LM Studio (from lmstudio.ai). Why use a model locally? Just for fun. Where we don’t have internet.
Did you know that you may block AI-related web-scrapers from downloading your whole websites and actually stealing your content. This way LLM models will need to have different data source for learning process! Why you may ask?
Asking BLOOM-560M “what is love?” it replies with “The woman who had my first kiss in my life had no idea that I was a man”. wtf?! Intro I’ve been into parallel computing since 2021, playing with OpenCL (you can read about it here), looking for maximizing devices capabilities.
Use Microsoft Azure AI Services to analyze images, voice, documents. No AI/ML or coding skills required. Responsible AI applies by EU AI act. Formerly Cognitive Services. Course of Action What is Microsoft Azure?
Wstęp Dzisiaj wyjątkowo będzie po polsku na temat ChatGPT. Zadałem sobie nieco trudu i wypróbowałem jego “możliwości” z zakresu programowania jako, że zasłyszałem o takiej opcji. Moim ulubionym w ostatnim czasie tematem jest OpenCL, zatem…. JA: to napisz mi kernel OpenCL który służy do sortowanie liczb zmiennoprzecinkowych Dostałem sortowanie bąbelkowe….
I’ve been playing around with several devices in a context of running OpenCL code on them. They have one common thing which is excessive heat coming out of GPU and heatsink being unable to dissipate it. I start with MacBookPro3,1. It has NVIDIA 8600M GT, which is known to fail.
Among various computing devices I have there is one that stands out it is NVIDIA Quadro NVS 140M because it supports only FP32 (float) operations, but not FP64 (double). It is generally too old. In OpenCL we have both pow function which takes double and float parameters.
You can put your #GPU in #Proxmox server box and pass thru computational power to virtual machines… just in case you would like to run your AI/ML things alongside your virtualized NAS 😀 Finally I got it working. I think so.
OpenCL is excellent in the field of numbers, but not that much into text processing. It lacks even basic functions available in regular C99. So the question is if it is worth trying to process some text in it.
OpenCL implementation od DES cipher is way faster than regular single-threaded C program. 1 mln encryptions take less than a second on RTX 3050 Ti, but also as much as almost 40 seconds on Intel i5-10200h single-thread application.
Today I came back to my OpenCL project (can be found here). I have had not tried it on my fresh Ubuntu 22 installation on my Dell g15 with RTX 3050 Ti. Altough I’ve got NVIDIA driver metapackage installed my program reported that there is some problem with shared library:
If you have OpenHAB on Proxmox or any other virtualization and it sometimes fails to grab RTSP stream and create snapshots, then there is high chance that everything is fine with the camera and network and the problem is within your server hardware.
On fresh installation of Ubuntu 22, using Opera for video playback can be an issue. So even after installing all things that you may think it could help – it does not work. The solution is to install chromium-ffmpeg and copy its libffmpeg.so library into Opera installation folder.
It is possible to make wire based motion sensor from Satel, wireles using Zwave network. Satel Opal, Opal Plus and Opal Pro have similar case which can hold additional module of Fibaro Smart Implant.
Recently we’ve ordered Fibaro’s HC3 as a replacement for OpenHAB. First because it has integration with Satel alarm system and second that it has native Zigbee support next to default Zwave. OpenHAB also has support for Satel and by using external adapter you can connect Zigbee devices too.
I tried to record video on my Dell G15 laptop using built-in camera. Unfortunately, by default it does not work. I tried Cheese and Webcamoid using several different settings. Cheese did not work with both built-in and external camera. Webcamoid worked with external camera only.
In order to integrate your SmartThings devices into Fibaro HC3 you need to create access token and review API documention which can be found here: https://developer.smartthings.com/docs/api/public#tag/Devices/operation/getDevices Then, to send on or off command to your device, grab its UUID, create QuickApp with the following code:
In popular DIY markets you can buy Philips WiZ LED light bulbs. They are affordable choice over Hue lights. WiZ lights operate thru WiFi connection and can be integrated easily within OpenHAB system as a wizlighting binding coming from marketplace.
If you want to run some curl on icon click from Ubuntu 22 panel, then you need to create file in ~/.local/share/applications: Then after few seconds, press special key (aka Windows key) and type your “application” name.
If you happen to have MacBookPro3,1 with Intel Core 2 Duo 2.2Hz DDR2 SDRAM with MacOS 10.8.5 on board then you probably wonder if there is a chance to access internet with modern browser. Yes, now you can with chromium downloader for chromium-legacy.
If you would like to control some electrical devices by switching them on and off then one possiblity is to use Rasberry Pi relay hat. It is from Waveshare/Botland and it fits on top of your Raspberry Pi. It contains 3 relays controllers with screw connectors.
I tried to compile my c++ code with c++11 features on GCC 4.2.1 which is default on my MacBookPro3,1 with MacOS 10.8.5. I used random library, so I needed newer version of GCC, 4.7 at least to be more precise.
To bring and old MacBookPro up to date you can use MacPorts. First you should download and install MacPorts package from macports.org. There is a pkg file to use specifically for Mountain Lion. That’s the easy one. A little harder would be to grab working XCode with command line tools.
I have this nice MacBook Pro (3,1) from 2007. It has Intel’s Core 2 Duo running 2.2 GHz. Unfortunately the battery is dead or electronics reponsible for charging it. It arrived fully charged and was working for like 2 hours or so.
Recently I got Lenovo ThinkPad T61 in working conditions. It has Intel Core 2 Duo T7100 @1.8 GHz, 4 GB of DDR2.
If you try using darktable 3.6 and previously been using older versions then you may be confused. In lighttable, import pane there are two buttons instead of one. Below you can find a little explanation. 1.
If you shoot RAW then most probably you need to have dedicated software for processing such files. I use darktable, however it is worth to mention that Windows Explorer and built-in image viewer can display RAW image. Just in case there is no other software available at the moment.
I started wondering if cheap YN lens can be compared with with some cheap Nikkor lens. Yes it can. I shoot same lens cap with Nikon D200 with these two lenses with are Yongnuo 50mm/f1.8 and Nikkor 35 – 70mm f3.3/4.5.
Some time ago I got rid of my D200. But now I have another one in excellent shape. In the past I upgrade from it to D7100 which is better in terms of resolution, sharpness and features.
It is a 24 megapixels digital SR camera. Sadly to say that I do not use this maximum resolution, because files are just too big. I even skipped using NEF raw files they are just enourmously huge for me. You can choose between 6 MP, 16MP and 24MP.
As previously stated, I tried to adapt SMENA 8M 40mm, f/4.0 lens to some DSLR. After week with no camera at all today I got my almost brand new NIKON D70s with only 1k pictures taken.
Continuing with USSR made SMENA 8M. I decided to take apart lens unit. It consist of upper cover with levers and lower unit with shutter blades, shutter spring-based mechanism and optics. There are three blades. They are mounted on pins located on the rotating plate.
I got this one as an exchange item for some old LCD TV. It is 35mm film compact camera made in USSR. Having quite a big monopoly on the eastern market, Soviet Russia manufacturer brought over 30 million of these to the markets. SMENA 8M is a very lightweight camera.
It seems to be very decent lens. Especially speaking of build quality which is glass and metal with very little rubber and plastic. It is prime lens at 300mm with constant f/5.6. It is quite light. Has 3 rings. First to adjust focus. Second to adjust aperture.
Made in Japan. This teleconverter fits on NIKON F cameras. It is a MF thing, however it has automatic aperture control as stated in its name “AUTO”. It increases focal length by the factor of 2. It came with a very elegant zipping pocket.
I was looking for NIKON camera that will allow me to use manual focus lenses with proper light metering. I found that D200 would suite this need. It a 10 megapixels camera with SONY ICX-483-AQA sensor. It seems that is the only camera using this one.
This would be my favorite and the only one digital camera, except one thing. It does not meter light on AI lenses, unfortunately. My copy already shot almost 50 000 pictures. It shows no wear at all, including battery.
This is one of my favorites. It is a NIKON F mount, AI lens with constant speed at f/2. It is a regular/normal lens. It has no AF, only MF. This copy come from between 1977 and 1979. It is clean inside. Focus and aperture ring operate very well.
As always I bought it for next to nothing. It is NIKON F mount lens. To be more precise it it a AI lens. No AI-S features. This review will be a little shorter than usually, because the lens is crippled by malfunctioning aperture blades.
Together with NIKON 75-150 f/3.5 and SOLIGOR 70-150 f/3.5 I recently purchased this one, which is NIKON F mount VIVITAR 70-210 telephoto lens with constant f/4.5. My main criteria did not change, so it should be reasonably priced and built. I thought that this would be a good one.
As many know, NIKON cameras and lenses have been heavily used by pros in the 1970-1980. They invest a lot in their gear. That is why the compatibility is important. You do not choose only a camera or lens, you choose complete system.
Recently I have been looking for some old style NIKON F AI MF lenses across the internet. I got this one. It is a discount, costs next to nothing. Of course it’s not only about the money, but about the performance you can get from it.
You can use either –query-gpu option: Example output: Or dmon: Example output: With little explanation:
Create dedicated replication user: Then add physical replication slot, in order to stay in sync with WAL.
Hello. I’m Michael, I’m IT professional and enthusiast. I graduated computer programming as well as economics at Szkoła Główna Handlowa in Warsaw. I’m author of 5 books concerning software design, development, quality assurance and performance.
No articles match. Try another word.
