In the past we have studied how to do limited number of concurrent tasks via buffered channels, an approach inspired from golang. This time we make use of native API of Semaphore to achieve the same goal.

We study how to embed value object into JPA entity class definition, and study how to auto-map our value object back to plain DTO object for the frontend

Record a docker-compose.yml to host a postgresql/mysql db locally. Also we clone existing DB by docker-command using postgres image

We use additional thread to execute computational task by built-in workers API

We study the asymptotic behaviour of n!

We study a special case where the repository.delete() can fail silently

We mention two common patterns in event-driven design for Spring boot that handles side effects due to domain events

A simple introduction to rtk-query

We record a method on how to reorder folders based on our predefined pattern

Record the study of standard concepts from nestjs while working in the current company.

We discuss how to remove a VPC completely when dangling ENIs remain there.

We study terragrunt via opentofu

We discuss programmatic way to git push changes and git commit empty commit for triggering github workflow.

We immitate the expo approach for all of our backend applications, here is how!

We try to build and develop an npm package.

Code to count the duration of a shell script

Record a standard setup as a transition from redux to zustand. This article will be kept updated along the way I delve deeper using zustand.

Record simple css to recover bullet points affected by tailwind package

Record method to load multiple env files in serverless framework.

Record a simple hook to block route traffic when updates have not been saved yet, and popup an alert to ask for confirmation.

Study cases when a lambda function need to invoke itself.

A record of simple excel to db-record task.

Simple scheduling to our springboot lambda function's endpoint

Google login without redirection of main page by popping up a new window.

Certain resources are not supposed to be recreated even we recreate our infrastructure via terraform. Let's study how we define and import database and s3 bucket separately.

Discuss how to make our terraform project as DRY as possible.

It is very common to have lamdba function being called by another function. This time we study two kinds of lambda functions to be invoked, one is snapstarted springboot lambda function, another one is an ordinary console lambda function.

Creation of modal is always a tedious task. We can create modal easily by ant-design's Modal component. But improper implementation can lead to plenty of highly non-reusable modals as the common mistake is to create that modal at the component we use it, making it tightly coupled with the state at which the modal is defined.

When business grows, a domain object becomes very bulky if all behaviour and the corresponding validation are added to the same entity file. We study code separation by the language feature of Kotlin using function literal with receiver.

We discuss a reliable approach to connect websocket api from api-gateway.

We discuss how to bring the power of caching/debouncing, the cahcing and cache-invalidatin, and also the handy booleans like isLoading of react query into world of react thunk actions.

We record how to transform all the schemas in swagger ui into typescript interfaces.

We record how to define gradle task to run npm script and how to ensure the gradle can recognize our npm binary

Record how to interact with mongodb in a JPA manner.

Record a standard step to alter the authentication method (choose which account to authenticate) when we push changes to a repository.

We record a rate-limited concurrent tasks techique borrowed from golang.

Record a simple migration script to remove all duplicated rows before adding a unique index to a column in a table.

This is a record of machineries in a springboot project for my own convenience.

We record the basic and elements of bubble tea.

Record a configuration to clean up old versions of lambda functions.

Record useful tricks in golang

Record an architecture to manage simple lambda functions that can be created directly in aws console.

Record the functioning configuration of usual resources.

Record a creation of a simple worker node using EC2 and K3S via terraform

After real project experience with AWS cloud, I revisit the knowledge in terraform in order to replicate the cloud infrastructure from UAT to PROD effectively.

We study the configuration of IAM roles and permissions to enable lambda functions to connect to RDS-Proxy which is a VPC-bounded resource

Record a simple script to create a presigned-url for file uploading in frontend

We study the basic of websocket-api from api-gateway.

We study the integration of snapStarted lambda with API-gateway.

Record the basic usage of react-query.

Let's get rid of the deprecated react beautiful dnd.

Let's get rid of the deprecated react beautiful dnd.

We record the use of ksp package that auto-generates DTO mapper for annotated entity classes.

We record a general purpose dialog whose width can be sm, md, lg, etc to avoid the consideration of width of the dialog content.

We record the use of ksp package that auto-generates DTO mapper for annotated entity classes.

Given that a domain is brought from third-party DNS provider, we study how to route the traffic to our cloudfront distribution.

Let's discuss how to orchistrate the interaction of lambda functions which are network-isolated, especially how to endow lambda functions with security group.

Suppose we want to test production-like environment by making a production build for mobile, it is helpful to host multiple backends in localhost and expose it via https, multiple ngroks can do the trick for us.

Setting eslint is not enough as people can still ignore it and commit their code. We introduce husky which provides a pre-commit hook to prevent unpassed code from being pushed.

We introduce simple functions for logging.

Due to the lambda-nature, snapStarted spring boot differs from ordinary spring boot application as it lacks the write permission to the /var/task of the lambda function.

We record LUSH and BRPOP in kotlin

We record a simple logger for our spring application

In testing we usually make use of in-memory database to separate the test data with our actual working database. For a long time people use H2 in spring boot for that purpose but that database is not eventually what the production uses, now we can align both!

We record useful postman script in manual testing.

Record miscellaneous detail of transitioning into Spring Boot as a node.js developer.

In the past we have introduced an applicationEventPublisher, this time we introduce decorators to further simplify the code!

We implement a doubly-linked list with generic data type that is useful when trying to group a set of "directional" objects.

We discuss how to create a lambda function that executes in docker container to get around the 250MB size limit.

Installing python and executing a python script with argument can be tedious to teammates who are not used to python. The same siutation can apply to all other languages, let's simplify the execution by doing it on github actions.

We study scheduled tasks by doing a regular backup of databases (pgsql + mongo) on a hourly basis.

There is occasion where we have created a pdf file and want to have an html version which looks exactly the same (e.g., pdf generated by LaTeX)

Let's discuss how to restore a mongodb from backup, how does the back up actually?

Let's discuss how to clone a database again.

We can design a button to determine what and when to do something instead of automating the actions for each push / merge request.

Working with existing database via jpa amounts to the need to manually model the table by entity classes. We study how to modify the data class generated by JOOQ into a "pre-entity" open class that can be inherited to define domain model.

We study the basic steps to enable inapp-purchase.

A simple implementation of distributive lock in Kotlin.

We have talked about monolithic DDD in the context of spring boot with the help of spring-provided ApplicationEventPublisher instance via dependency injection. This is not unique to spring and let's bring it into nodejs in the express framework.

We study how to effectively deliver request to non-blocking coroutine scope and release that thread for other request.

A Stripe event is always delayed, and some system relies on database persistent change by Stripe events. We study how frontend can wait for the event before fetching latest data.

Study the syntax for function literals which helps construct DSL-like syntax.

We record how to adjust a stripe subscription in code using kotlin.

build.gradle.kts is like a Makefile with powerful feature that you can write kotlin code on it, let's study useful customization.

Run a spring boot in docker image using lambda function

Traditionally lambda function is as simple as running the function defined in a zipped package. But when dependencies get complicated and when file size inevitably exceeds 250MB limit when being unzipped, we need to consider using docker image as an alternative.

We record a two-stages dockerfile to build a docker image of spring boot application.

Record convenient tool to manage time in kotlin

There are two ways for DDD Project in spring boot: DDD using ORM and that with ORM, let's discuss the one without ORM in this article

Record a two-stage docker file for building springboot application.

Record the common usages of JOOQ library.

Record a component for simple reuse.

When creating a transaction we need to wait for the checkout-session-completed event in order to execute follow-up action in our own system, we relies on metadata to follow the transaction.

We record how to require a saga to wait certain amount of time before dispatching cancelling events, even our server restarts.

We record useful query for debugging of domain events with json payload.

We study some example of deserialization of json strings using Gson and kotlinx-serialization-json.

The generation of the bean of WebMvcConfigurer suffices to provide a global configuration of CORS via Rest Controller, but that fails to configure non-mvc requests (such as those intercepted by our interceptors). Let's study an alternative.

We record how to use interceptor and custom annotation to record the flow of commands and events.

We record how to introduce reusuable interceptor in kotlin.

We study how to send email again in kotlin.

Study the error handling for Subscription Query.

Record the minimal code template for KotlinLogger.

Record the usage of react-router v6.

Record a minial working example for react-beautiful-dnd.

Sometimes it is much more convenient to save multiple config files in a repository than to git-ignore it. We study the encrpytion of those files using git-crypt.

Record a standard procedure to create an react-app via vite.

It is easy to make input field, but building one in chatroom is a bit different.

We continue the previous study on k8s, this time the EKS.

We continue the previous study on k8s, this time the networking.

We study the basic objects in the world of k8s.

Study a simple query function WIDTH_BUCKET to obtain a set of time-series data and study nice chart library to generate a bar-chart with multi-columns.

A JWT authentication should be simple and straight-forward. Let's forget about spring-security and start with the basic idea.

We try to implement a JWT authentication and study how to construct custom token payload for the backend.

This is a study on gradle to help understand how to manage a project by modules. In java this helps separate the dependencies precisely, i.e., no one can access resource from incorrect layer. This makes creating unit tests much more easily (in case you have written queries directly in controller layer, you know what I mean).

Config a global setting of how the error should return to frontend.

We try to throw an exception inside a transaction to see whether a rollback takes place.

Record a simple setup for event emitter and listener.

Record the setup of a spring boot project with JOOQ.

Record SQL script for managing db user persmissions

Just record a zip file that is customized for my own use.

We try to get the full list of type hints in vscode instead of the shortened and incomplete ones.

We record a github workflow for deploying serverless backends

We record how to download a complete record of cloudwatch logging inside a log-stream.

Record some mistake that can be avoided when designing a database schema.

In the past we have studied CDC with the help of confluent here. This time we host everything locally to prepare ourselves to host CDC without the dependency on confluent.

We discuss the technical detail of OTA (Over The Air) update which avoid unnecessary rebuild for updating a mobile application.

Basic CRUD with DynamoDB

We study how to create stack of aws resources and how they can interact with each other.

We record a standard procedure to create a custom layer for 3rd party libraries.

We study the concurrency models in lambda functions to avoid unexpected failure.

Sometimes a library may require user installing gcc, c++, ca-certificates, or some other linux-specific libraries. Let's use docker image to provide us a consistent working environment.

We study how to configure Debezium in Confluent to listen changes of database, which is very helpful in streaming real-time changes to frontend.

We study read lock and write lock.

Study how to share a SINGLE channel to all goroutines and how to close it properly.

We implement a simple hashtable by using golang.

We record how to implement in-app notification which consequently makes caching much more effective and flexible.

The concept of message queue is easy but writing them can easily be messy due to boilerplate code, we summarize how to organize them into a MessageQueue class.

We record how to make custom script with custom predefined variable in command line via running js script.

Convenient simple functions to set as default in SQL.

Some simple yet common use case of as-with clause in sql, and its counterpart in kysely.

The default Datetime object in PGSQL by default is UTC+0. Application-wise we wish an absolute timestamp that is compatible with frontend (i.e., javascript), here is how.

We record how to use ULID instead of UUID in PostgreSQL

We record how to mark a manually changed sql file as a successful migration in prisma.

Record the commonly used migration scripts in prisma and sql.

Record some exmaple how to declare types from library that is untyped.

We record how to implement push-notification in both Android and iOS.

We study how to organize code for redis caching in a nodejs project

We introduce how to alt+1, alt+2, ... to switch among different vscode window.

We study how to automate the deployment process with containerized backend image.

In the past we have discussed algolia backend and the corresponding frontend, revisit this topic with all code written in nodejs

Fundamental and basic use of github actions.

Record basic scriping for trigger.

Record the procedure the reverse-engineer a PGSQL database.

Record minimal code needed to use prisma as just a table migration tool.

We record a vscode configuration to customize where to save our image when pasting image in md files with automatically generated url inside markdown file.

Record the implementation of various queues and integrate them with springboot.

Record how to experiment with the sqls generated by query builder.

We record the config for a starter project in springboot that mimics the workflow of prisma+kysely in nodejs world.

In recent mui makeStyles is no longer maintained, but css in tss is still a very good idea so we record an alternative in this article.

Revisit the basic of springboot application.

Record and introduce the workflow of using prisma and kysely.

Record the actual implementation in frontend for fail > refresh access-token > retry.

Record the steps to run load and stress test.

A list of sql script for copy and paste.

Methods of sharing environment variabls has always been being diversified. Some may just keep the .env in the project. Some may share it via instant messagers. Some may store it in a webpage that require a login. We share a method that should be a norm for intense aws users!

We build smaller docker image by copying the only used components into new docker image

We record how to create a complete web application project in go.

We record how to make use of the native gmail api to send email without sendgrid which cost money with only 200 mails a date in free plan (while gmail api is free).

Record the workflow used in database migration, integrated with tools like dbdiagram.io.

Explore transactions with go and raw SQL

Record a pan guesture component that can swipe an item to show hidden buttons

Record the standard qureis in SQL.

Containerization of vanilla JS project is simple, with typescript we need little more steps.

Record my custom configuration of express logger

Just a syntax record

We record a workflow of using goose and sqlc to work with changes of database schema.

In the past we have discussed SSE by kind of short polling in the backend (keep looping to see whether a key has message to pop out in redis queue). This time we send messages to frontend by listening subscriptions on EventEmitter, an approach very native to languages in which channel is implemented, like go and rust.

Though we are able to create a bottom-sheet on our own, we may as well use existing stable ones from others!

We implement a reusable utility class that handle most of the jobs in creating excel programmatically.

We discuss How to keep running tasks even when our app is in background state (including screen being locked.

How to make a modules and finally, we can apply pre-made modules from others!

Let's split our code in multiple terraform files.

Standard technique to create multiple resources by conditions and loops.

We discuss how to manipulate variables (dev, prod, etc...) and plan a small section on remote execution of shell scripts once an instance is launched.

A study of Terraform basic building blocks.

Pin to zoom is very common but it is not an out-of-the-box feature for camera API in iOS. Let's bring it back on our own.

An introduction to react-native-audio-recorder-player

There are two ways of deploying tasks in ECS fargate, by either "create service" or "run task", let's get hands on experience with both methods.

File streaming is a very basic technique to effectively transmit files from frontend to backend and, of course, from within backends as well.

We build our own bottom sheet.

We build our own toast meassages.

Record detail of expo-cli workflow in development build.

We can create our custom component by creating custom behaviour via customizing animtation!

We create a special collection and a hook to the save method of a collection to obtain an auto-incremented id field!

We create a voice chat application on mobile using third party API called AgoraRTC.

We create an aggregation pipeline which acts like a left-join.

We study how to scale up the chat services horizontally by Redis.

Study of Tokio by building a chat server.

Basic reivew of API provided by socket.io-client and socket.io in nodejs.

Testings not only justify our functions are working, it also demonstrates how our function is used. We will be setting up tests in typescript.

Package for combining env files conveniently.

Record a configuration for auto-formattting.

Let's define operator overloading on finite field Z/pZ for prime p.

In the browser our route consist of many information, like messageId, emailId, etc, which enables our web page to select correct data based on the route. In this post we discuss how to effectively construct such url with type safty under a RouteSchema.

This is a beginner notes.

Record how to implement CORS in express to allow certain origins to get access instead of allowing all origins

We implement file downloading feature in a get/post request in both express and spring, we handle them in memory and therefore no disk i/o is needed.

An introduction to how to effectively create SSE event mimicing the single-thread event loop adopted by nodejs in springboot, and how do we actually implement it in nodejs as well.

Record a hook to listen to scrolling-up and scrolling-down events, and an easy way to inject our callback to this events using this hook.

Record a list of box-shadow in a json object.

A simple component and function that helps rerender a component effectively.

A method to pass complicated function in a component to its parent.

A simple backend that perform google authentication. I personally use this to restrict users who can access my project.

I used to use useRouteMatch in react-router-dom v5, which has been changed completely in v6 into something called useMatch or useMatches. We talk about building the param searching function on our own to get rid of these unhandy "black boxes".

Record a configuraton for snackbar utils.

Record a multi selection component.

We get rid of the headache of writing mongo agggregation using mongo apis in java given that we already know how to write that in json format.

Record a hook for determining whether click have happened outside of our target dom element.

We introduce a service called Algolia which provides an easy search engine integration that helps build quick and accurate search functionality.

Guide to creating serverless flask and express application.

A introduction of OAuth2 Consent Setting and Pubsub for publishing update of gmail inbox.

A full breakdown of steps deploying a react project to s3 with SSL encryption.

A full breakdown of steps deploying a react project to s3 with SSL encryption.

Record the detailed implementation of lazy loading.

normalizr will be helpful if we have complex manipulation of data in an array (like drag and drop)

We list the standard procedures to debug a dockerized spring application.

Record CICD script in gitlab for deploying react pages.

We record a command to run an image in non-stop mode for debugging.

Record the latest tsconfig.json I use.

Revisit the fundamentals of docker and the related gitlab-ci workflow.

Record the use of redux-persist for persisting data in redux store.

Record the implementation of radio button group and dropdown list, in a hope that we don't need to waste time cooking it up again in the future.

Record CSS that makes scollbar look better.

We list sample usage of createThunkAction provided by redux-toolkit in order to single out the logic of data-fetching away from the UI component.

Just record the usual API that we may need in using dayjs.

We demonstrate it by examples.

$lookup only works for the whole collection, we demonstrate how to left join another collection using a field which is an array.

Record my template of redux slice.

Record the mongo query using aggregation pipeline to get the results as in the title in my usual work. These include $lookup, $project, $arrayElemAt, {$gt: {$size: 1}}, etc interesting operations.

Record how to write CSS animation with keyframes in mui makeStyles.

Click and then get detail is a very routine practice for data scrapping. We record how to do that by selenium in python.

We record the whole workflow of uploading file from react frontend to springboot backend.

We study how to register a custom url protocol to launch our desktop application.

In typescript we have type A = B | C, we also have an analogue in C++.

In the course of translating pytorch model into libtorch model there are traps and tricks that are worthing being recorded. Also record the simple use of opencv as it substitutes the role of numpy in python.

Previously we have mentioned how to use precompiled header in visual studio project, this time we record how to do it in cmake project

Sometimes the constant CMAKE_CXX_STANDARD does not guarantee the C++ standard we use in compilation. We add a line to guarantee which target is compiled in which C++ standard.

Record the flow of including opencv and libtorch into CMake project.

Beginning study of diffusion model.

Discuss how two threads communiate with each other.

Record useful utility functions that I have found during my project on desktop app in C++.

Fix the error of failing to find source files when it really can for cmake projects.

We discuss how to import models trained in pytorch into cpp project.

We create a simple TCP Server via winsock.

Record some experience and the CMakeLists.txt's that I have used.

Record the usage of gray-matter.

A simple TCP server that receive and hold a connection.

Record a useful print function which behaves like console.log in javascript and print in python.

We try to implement iterator that iterates elements inside our vector class in the PREVIOUS BLOG.

We can avoid heap memory allocation by using C style array stored in stack memory, given that we don't need such a flexibility of dynamic array (the std::vector class) and the size of the target array is known beforehand.

Record standard procedure to make a precompiled header file to avoid rebuilding it every time we build the whole project.

We implement our string class and demonstrate how to create a copy constructor for deep copying a string.

Record solution setup for cleaner folder structure

The latest react mui version (v5.0) has breaking changes that make the config files in THIS POST fail. We discuss the corresponding necessary changes.

Record the setup for redux toolkit.

Record how to change pytorch model into onnx model and deploy it to frontend.

Record how to change pytorch model into onnx model and deploy it to frontend.

Record the study of headless detector for face and landmarks.

Record the most recently used combination of data augmentations.

Discuss how to customize the patching of the results from __getitem__ method of Dataset object in pytorch.

Vscode task.json for using 2020 standard.

Code implementation in pytorch on Cascade RCNN, the code base is mainly a modification of my previous project.

Record an attention-based component.

Record a script to clip gradient to avoid graident explosion.

Describe how to fed the features from Swin Transformer into Faster RCNN.

In practice we can directly use torchvision.models.swin_t to use pretrained model. The Reimplementation of the model is for sheer purpose of my self-learning.

Record a trick to create mesh grid coordinate.

Record the study of the basic mechanism how a transformer perform classification task on images.

Record the basic syntax in learning angular.

Continuation of the previous blog post on transformer, discuss more modern pipeline for training a transformer (or any NLP task in general) using the latest torchtext.

Record the basic building block and structure of a Transformer. The main reference of this article is this blog post.

Sometimes a loss becomes nan in rare occasion (e.g., it occurs once per 5~7 epochs), in this case we record a script to restart the training using the latest weight.

Different backbones have different APIs for extracting features from an image, we record some of them.

Record my data augmentation used in object detection.

Minimal functioning implementation of faster rcnn (with and without fpn).

Break down the code for my own study. e.g., the way it does image cropping, resizing, etc augmentations while keeping the vertice consistent are very valuable reference, the way it does nms is also succint! Source Code: Link

Record usual api for drawing apis in cv2, pillow and shapely.geometry.

Mask in numpy, tensorflow and pytorch is very useful to filter out desired values, but sometimes the index that the mask marks as True is also import, we record a function to retrive those information.

Record the use of a progress bar when using for loop, and record how to customize the output.

Create a console log that freeze the position but keep updating the numerics in training.

Record useful tools and commands in pytorch which I learn from translating DefectGAN model in tensorflow into pytorch.

Record the basic knowledge we need to know in shell script.

Record standard step to set up tensorflow-gpu, cuda and cudnn.

Some implementation on DefectGAN.

We discuss the rigorous proof of Singular Value Decomposition.