分类 开发 下的文章

Tensorflow-lite

https://tensorflow.google.cn/lite/guide/build_cmake?hl=zh-cn

Download git - tensorflow

cd ~
mkdir tflite-micro
cd tflite-micro

git clone https://github.com/tensorflow/tensorflow

cd tensorflow

Install deps

  • Ubuntu
sudo apt-get update
sudo apt-get install git build-essential cmake python3 python3-pip
  • macos
brew install git cmake python

Download & instal Bazel

  • 安装 Bazelisk(自动管理 Bazel 版本)
curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.17.0/bazelisk-linux-amd64 -o /usr/local/bin/bazel
chmod +x /usr/local/bin/bazel
  • [推荐] 手动管理版本

首先进入 tensorflow 目录 查看 .bazelversion

7.4.1

比如 目录配置是 7.4.1 那么,就打开 bazel - git https://github.com/bazelbuild/bazel/releases 找到对应的版本,如

  • macos

https://github.com/bazelbuild/bazel/releases/download/7.4.1/bazel-7.4.1-darwin-arm64

  • linux

https://github.com/bazelbuild/bazel/releases/download/7.4.1/bazel-7.4.1-linux-x86_64


tflite-micro

1. download tflite-micro

git clone https://github.com/tensorflow/tflite-micro
cd tflite-micro

2. download the suitable version of bazel

cat .bazelversion

# result like 7.0.0

go to https://github.com/bazelbuild/bazel/releases

and download the bazel binary package

mkdir ~/tools/bazel/7.0.0
cd ~/tools/bazel/7.0.0

wget https://github.com/bazelbuild/bazel/releases/download/7.0.0/bazel-7.0.0-linux-x86_64

chmod +x bazel-7.0.0-linux-x86_64
# create link
ln -s bazel-7.0.0-linux-x86_64 bazel

set the env

vi ~/.bashrc

or

vi ~/.zshrc # macos
# bazel_env_start
export PATH="~/tools/bazel/7.0.0:$PATH"
# bazel_env_end

flush the rcfile

# bash
source ~/.bashrc

# zsh
source ~/.zshrc

Now the env setting is done.

check the version of cmd bazel

bazel --version

# terminal shows below
bazel 7.0.0

Spark是用于分布式数据管理和处理系统,源于google的mapreduce系统以及开源社区Hadoop,并对其做了诸多补充和升级,譬如采用内存来管理数据,大幅度提高了响应速度,更适合现代的各类分布式数据场景,如实时大数据交互、人工智能、空间数据处理等

以下是在本机创建用于验证和个人部署spark的简要步骤:

macOS

brew install apache-spark

安装完成后,spark会将安装在 /opt/homebrew/Cellar/apache-spark/$version 下,比如4.0.1 就在 /opt/homebrew/Cellar/apache-spark/4.0.1

我们将路径输出到环境变量中,编辑 vi ~/.zshrc,不用zsh的需要编辑其他rc文件

在后面添加

# spark_env_start
export SPARK_DIR="/opt/homebrew/Cellar/apache-spark/4.0.1"
export PATH="$SPARK_DIR/bin:$SPARK_DIR/libexec/sbin:$PATH"
# spark_env_end

编辑完成后重新打开terminal 或者 source ~/.zshrc 完成安装和环境配置

Ubuntu

安装JDK, ...

建议使用Scala 3.x,相应的jdk版本建议使用11,如果安装Scala 2.x的话 需要安装jdk@8

sudo apt install default-jdk

安装others

ubuntu

sudo apt install scala -y

安装spark

首先打开spark官网下载页 https://dlcdn.apache.org/spark/

选择一个版本,比如 3.5.7

复制对应的下载地址,如 https://dlcdn.apache.org/spark/spark-3.5.7/pyspark-3.5.7.tar.gz

创建一个目录,比如 ~/spark

mkdir -p ~/spark
cd ~/spark

下载

wget https://dlcdn.apache.org/spark/spark-3.5.7/spark-3.5.7-bin-hadoop3.tgz

解压并进入文件夹

tar -xf spark-3.5.7-bin-hadoop3.tgz -C ./
cd spark-3.5.7-bin-hadoop3
ll #查看目录 是否有 bin sbin

total 96
drwxr-xr-x 1 shezw shezw   170 Sep 18 04:52 ./
drwxr-xr-x 1 shezw shezw   100 Oct  5 15:27 ../
-rw-r--r-- 1 shezw shezw 22916 Sep 18 04:52 LICENSE
-rw-r--r-- 1 shezw shezw 57842 Sep 18 04:52 NOTICE
drwxr-xr-x 1 shezw shezw     6 Sep 18 04:52 R/
-rw-r--r-- 1 shezw shezw  4605 Sep 18 04:52 README.md
-rw-r--r-- 1 shezw shezw   166 Sep 18 04:52 RELEASE
drwxr-xr-x 1 shezw shezw   748 Sep 18 04:52 bin/
drwxr-xr-x 1 shezw shezw   288 Sep 18 04:52 conf/
drwxr-xr-x 1 shezw shezw    68 Sep 18 04:52 data/
drwxr-xr-x 1 shezw shezw    14 Sep 18 04:52 examples/
drwxr-xr-x 1 shezw shezw 13296 Sep 18 04:52 jars/
drwxr-xr-x 1 shezw shezw    32 Sep 18 04:52 kubernetes/
drwxr-xr-x 1 shezw shezw  2402 Sep 18 04:52 licenses/
drwxr-xr-x 1 shezw shezw   338 Sep 18 04:52 python/
drwxr-xr-x 1 shezw shezw  1030 Sep 18 04:52 sbin/
drwxr-xr-x 1 shezw shezw    56 Sep 18 04:52 yarn/

其中 bin, sbin是存放 可执行与服务启动文件的目录,需要配置到系统环境变量中,使用pwd获取到当前目录,如 /home/shezw/spark/spark-3.5.7-bin-hadoop3

编辑 vi ~/.bashrc 在最后添加

# spark_env_start
export SPARK_DIR="/home/shezw/spark/spark-3.5.7-bin-hadoop3"
export PATH="$SPARK_DIR/bin:$SPARK_DIR/sbin:$PATH"
# spark_env_end

编辑完成后重新打开terminal 或者 source ~/.zshrc 完成安装和环境配置


使用 start, stop来开启和关闭spark

start-master.sh
stop-master.sh

启动完成后,可以通过

localhost:8080 来访问spark的web页面,其中也会显示服务的端口号,一般是7077

1. 安装 Conda

如果你还没有安装 Conda,可以通过以下两种方式之一进行安装:

Anaconda:一个包含 Conda、Python 和许多常用数据科学库的发行版。你可以从 Anaconda 官网 下载并安装它。

Miniconda:一个更轻量级的发行版,只包含 Conda 和必要的 Python 组件,你可以使用 Conda 安装其他库。可以从 Miniconda 官网 下载并安装它。

2. 创建新的 Conda 环境

为了隔离项目依赖,最好为每个项目创建一个新的 Conda 环境。使用以下命令创建一个新的环境:

conda create -n my_ml_env python=3.12

这里 my_ml_env 是你为环境起的名字,python=3.12 指定了 Python 的版本。你可以根据需要选择不同的 Python 版本。

3. 激活环境

创建环境后,使用以下命令激活环境:

conda activate my_ml_env

4. 安装机器学习库

在激活的环境中,你可以安装所需的机器学习库。Conda 允许你从 Anaconda 仓库安装库,这些库已经为科学计算进行了优化。以下是一些常用的机器学习库:

NumPy:基础数学库。 Pandas:数据分析库。 Matplotlib:绘图库。 Scikit-learn:机器学习库。 TensorFlow 或 PyTorch:深度学习库。 安装这些库的命令如下:

conda install numpy pandas matplotlib scikit-learn
conda install tensorflow  # 或者 conda install pytorch

5. 管理环境

查看所有环境:

conda env list

退出环境:

conda deactivate

删除环境:

conda env remove -n my_ml_env
  1. 更新和升级 更新环境中的所有包:
conda update --all

升级 Conda:

conda update conda

6. 使用 YML 文件管理环境

Conda 允许你使用 YML 文件来管理环境,这样可以更容易地复制和共享环境配置。创建一个 YML 文件,例如 environment.yml,并添加以下内容:

name: my_ml_env
dependencies:
  - python=3.8
  - numpy
  - pandas
  - matplotlib
  - scikit-learn
  - tensorflow
  - jupyter

然后,使用以下命令创建环境:

conda env create -f environment.yml

这将根据 YML 文件中的依赖关系创建环境。

通过这些步骤,你可以使用 Conda 为机器学习项目创建一个完整的开发环境,包括必要的库和工具。Conda 的强大之处在于其能够管理复杂的依赖关系,并且可以轻松地在不同环境之间切换。

patch libs

ref

    1. install patch-package
npm i patch-package --save-dev
    1. modify the source code in third party libs
    1. generate the .patch
npx patch-package @mantine/core
✔ Created file patches/@mantine+core+7.2.2.patch

remove old source and reinstall

rm -rf node_modules/@mantine/core
bun install 
// npm install

jest

jest can not found canvas

npm i jest-canvas-mock --dev

and add 'canvas' : 'jest-canvas-mock' to moduleNameMapper in jest.config.js.

jest css, svg ...

module.exports = {
    "roots": [
        "<rootDir>",
        "/Users/shezw/dev/web/react/easyvgl-editor"
    ],
    "modulePaths": [
        "<rootDir>",
        "/Users/shezw/dev/web/react/easyvgl-editor"
    ],
    // "moduleDirectories": [
        // "node_modules"
    // ],
    moduleNameMapper: {
        'canvas':'jest-canvas-mock',
        "@mantine/core/styles.css": "<rootDir>/node_modules/@mantine/core/esm/index.css",
        // "@mantine/core/(.*).css": "<rootDir>/node_modules/@mantine/core/esm/$1.css",
    },
    transform: {
        "^.+\\.[t|j]sx?$": "babel-jest",
        "^.+\\App.css$": "jest-transform-css",
        ".+\\.(styl|less|sass|scss|png|jpg|ttf|woff|woff2|svg)$": "jest-transform-stub"
    },
}

jest test throw case

Use a clouse function wrap the test case and check if the result throw an Error.

test('Convert Color to RGBA', ()=>{

    const { toRGBA } = colorConvertor();

    expect( ()=>{ toRGBA("#333444",1000) } ).toThrow(RangeError);
    expect( ()=>{ toRGBA("#333444",-1) } ).toThrow(RangeError);
    expect( ()=>{ toRGBA("#331",50) } ).toThrow(Error);

    expect( toRGBA('#ff3366',100) ).toBe('ff3366ff');
})

jest wrong environment

    The error below may be caused by using the wrong test environment, see https://jestjs.io/docs/configuration#testenvironment-string.
    Consider using the "jsdom" test environment.

https://jestjs.io/docs/configuration#testenvironment-string

https://jestjs.io/docs/tutorial-react

npm install --save-dev jest babel-jest @babel/preset-env @babel/preset-react react-test-renderer

Jest test fails with "window is not defined"

add globals window to jest.config.js

module.exports = {
    globals: {
        "window": {}
    }
}

TypeError: Cannot read properties of undefined (reading 'html')

npm install --dev jest-environment-jsdom

jsdom environment

    The error below may be caused by using the wrong test environment, see https://jestjs.io/docs/configuration#testenvironment-string.
    Consider using the "jsdom" test environment.

    ReferenceError: document is not defined
/**
 * @jest-environment jsdom
 */
import '../jest.mock'
import React from 'react';
import {cleanup, render, screen} from '@testing-library/react';
import App from './App';


afterEach(cleanup);
test('Render App', () => {
    // ...
}

Class constructor Stage cannot be invoked without 'new'

use ts-jest instead of jest


TypeError: window.matchMedia is not a function

https://jestjs.io/docs/manual-mocks#mocking-methods-which-are-not-implemented-in-jsdom

src/jest.mock.js

Object.defineProperty(window, 'matchMedia', {
    writable: true,
    value: jest.fn().mockImplementation(query => ({
        matches: false,
        media: query,
        onchange: null,
        addListener: jest.fn(), // deprecated
        removeListener: jest.fn(), // deprecated
        addEventListener: jest.fn(),
        removeEventListener: jest.fn(),
        dispatchEvent: jest.fn(),
    })),
});

xx.test.tsx

import '../jest.mock'

Sometimes, we need to run a script to modify saveral files at once.

We can use xargs on shell tube.

ls | xargs echo 

As the command before, xargs run echo and use the result from ls.

this example shows how to use xargs modify all index.html that change *.js to *.js?ver=xxx

verStr="1.0.0"

# set version for  .css .js files
find -name "index.html" | xargs -i cp {} {}.bak 

find -name "index.html" | xargs sed -i 's@\.js.*\"@.js?ver='${verStr}'\"@g'

# reset files
# find -name "index.html" | xargs -i mv {}.bak {} 

在进行嵌入式开发的时候,文件传输是比较麻烦的 在调试过程中,最方便的方式是使用nfs进行文件目录共享 记录一下在设备和ubuntu之间共享的方法

在ubuntu上开启nfs共享

1. 安装和开启nfs-server服务

sudo apt install nfs-kernel-server

2. 配置共享文件目录

此时可以创建一个新目录 或者使用 开发目录来进行共享,这里以 /home/my/dev 为例

sudo vi /etc/exports

添加

/home/my/dev  192.168.*(rw,sync,no_root_squash)

如果出现Protocol not supported 可以在 /etc/default/nfs-kernel-server末尾添加

PRCNFSDORTS="--nfs-version 2,3,4 --debug --syslog"

3. 刷新服务

sudo systemctl restart nfs-server.service

4. 检查状态

sudo exportfs -v

daisyUI

https://daisyui.com daisyUI — Tailwind CSS Components WebdaisyUI is a plugin for Tailwind CSS. It works on all JS frameworks and doesn't need a JS bundle file. Install daisyUI as a dev dependency and use the class names just like any …

Chakra UI

https://chakra-ui.com Chakra UI - A simple, modular and accessible component … WebCreate accessible React apps. with speed. Chakra UI is a simple, modular and accessible component library that gives you the building blocks you need to build your React applications.

mantine (UI)

https://mantine.dev/

Global web icon

react-icons.github.io https://react-icons.github.io/react-icons React Icons - GitHub Pages WebReact Icons. Include popular icons in your React projects easily with react-icons, which utilizes ES6 imports that allows you to include only the icons that your project is using. …

tabler-icons

https://tabler-icons.io/

Over 4850 pixel-perfect icons for web design Free and open source icons designed to make your website or app attractive, visually consistent and simply beautiful.

Introduction to Immer

GitHub Pages https://immerjs.github.io › immer Immer (German for: always) is a tiny package that allows you to work with immutable state in a more convenient way. ‎React & Immer · ‎Using produce · ‎Supporting immer · ‎Installation

hooks

immer

https://zhuanlan.zhihu.com/p/639089491

  1. use-immer库的高级用法 下面我们将介绍use-immer库的高级用法,包括:

3.1 使用produce函数 produce函数是use-immer库中最重要的函数之一。它可以帮助我们更加方便地管理和更新JavaScript对象和数组。下面是一个简单的例子:

import { produce } from 'immer';
const state = {
  todos: [
    { id: 1, text: 'Learn React', completed: true },
    { id: 2, text: 'Learn use-immer', completed: false },
  ],
};
const nextState = produce(state, draftState => {
  draftState.todos.push({ id: 3, text: 'Learn Redux', completed: false });
});
console.log(nextState.todos);

在上面的例子中,我们使用了produce函数来更新state中的todos数组。我们可以看到,使用produce函数可以帮助我们更加方便地更新JavaScript对象和数组。


react snippets

a quick way to create a code template for a component or etc...

rcc

refs

ref is a reference to a element but ref maybe null

react-hook-form

transform several properties

const obj = {
    className: "none",
    size: 14,
    color: "red"
}

<Comp
    { ... obj }
/>

equals

<Comp
    className={obj.className}
    size={obj.size}
    color={obj.color}
/>

... means    loop the obj and set each key to the value

zod , joi , ... validation libs

import { z } from 'zod';

// create the limites
const schema = z.object({
    name: z.string().min(3, {"message":"at least 3 characters."}),
    age: z.number({invalid_type_error:"required"}).min(18) // the api shoud be mobile and ...
})

// create the  interface for data
type FormData = Formz.infer<typeof schema>;
@hookform/resolvers
const {
    register,
    handleSubmit,
    formState: {errors,isValid}
} = 
useForm<FormData>( { resolver: zodResolver(schema) } );
useEffect()

useEffect is similar to a "OnRendered" Callback when JSX modules is rendered then react call this function

finally()

the finally method will be called both succeed or failed when fetch on axios

?. optional link

seemly to swift syntax

obj?.type means obj && obj.type

Abort Signal

Mozilla Developer https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal AbortSignal - Web APIs | MDN WebAug 4, 2023 · English (US) AbortSignal. The AbortSignal interface represents a signal object that allows you to communicate with a DOM request (such as a fetch request) and abort it …

const controller = new AbortController();
const signal = controller.signal;

const url = "video.mp4";
const downloadBtn = document.querySelector(".download");
const abortBtn = document.querySelector(".abort");

downloadBtn.addEventListener("click", fetchVideo);

abortBtn.addEventListener("click", () => {
  controller.abort();
  console.log("Download aborted");
});

function fetchVideo() {
  fetch(url, { signal })
    .then((response) => {
      console.log("Download complete", response);
    })
    .catch((err) => {
      console.error(`Download error: ${err.message}`);
    });
}

Array.reduce()

Mozilla Developer https://developer.mozilla.org/.../Array/Reduce Array.prototype.reduce() - JavaScript | MDN - MDN Web … WebSep 14, 2023 · Array.prototype.reduce () Try it. The reducer walks through the array element-by-element, at each step adding the current array value to the... Syntax.

JavaScript Array filter()

WebThe filter () method creates a new array filled with elements that pass a test provided by a function. The filter () method does not execute the function for empty elements.

lv_draw_sw_img_decoded

img_decoded 是用于解码图片并且将图片绘制到指定buffer区域。

LV_ATTRIBUTE_FAST_MEM 
void lv_draw_sw_img_decoded
(
struct _lv_draw_ctx_t * draw_ctx,
const lv_draw_img_dsc_t * draw_dsc,
const lv_area_t * coords, 
const uint8_t * src_buf, 
lv_img_cf_t cf
)

/**
    struct _lv_draw_ctx_t * draw_ctx        // 绘制上下文
    const lv_draw_img_dsc_t * draw_dsc      // 图像素材绘制描述
    const lv_area_t * coords                // 绘制区(图像在output buffer上的绝对定位)
    const uint8_t * src_buf                  // 图像buffer
    lv_img_cf_t cf                          // 图像色彩格式
**/

绘制上下文中,与当前绘制图像相关的属性:

typedef struct _lv_draw_ctx_t  {
    /**
     *  Pointer to a buffer to draw into
     */
    void * buf;

    /**
     * The position and size of `buf` (absolute coordinates)
     */
    lv_area_t * buf_area;

    /**
     * The current clip area with absolute coordinates, always the same or smaller than `buf_area`
     */
    const lv_area_t * clip_area;

} lv_draw_ctx_t;

与图像本身的绘制效果相关的:

typedef struct {

    int16_t angle;
    uint16_t zoom;
    lv_point_t pivot;

    lv_color_t recolor;
    lv_opa_t recolor_opa;

    lv_opa_t opa;
    lv_blend_mode_t blend_mode : 4;

    int32_t frame_id;
    uint8_t antialias       : 1;
} lv_draw_img_dsc_t;

在进行macOS嵌入式调试的时候,用gdbserver报错 使用pyenv安装2.7之后同样无果,后使用安装包安装解决

dyld[675]: Library not loaded: /System/Library/Frameworks/Python.framework/Versions/2.7/Python
  Referenced from: <26C16900-9C5F-32FC-9593-0458A2B42E1C> /Users/mac/dev/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-gdb
  Reason: tried: '/System/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file, not in dyld cache), '/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file)

copied from: https://blog.shahednasser.com/how-to-install-python-2-on-mac-12-3/

As of macOS 12.3+, Python 2 which was installed by default at /usr/bin/python have been removed. This can lead to a lot of issues for developers that need to use Python 2 or that use other tools that depend on it.

Install Python 2 To install Python2 again, simply download the macOS universal installer from Python's website

After downloading the installer run it to install Python 2. This will automatically install Python in /usr/local/python as well.

Test it Out You can run Python 2 in your terminal now and other tools using it will no longer have issues:

python

lvgl doc flex

lv_obj_set_layout(obj, LV_LAYOUT_FLEX)

The possible values for flex_flow are:

LV_FLEX_FLOW_ROW: Place the children in a row without wrapping
不换行的ROW

LV_FLEX_FLOW_COLUMN: Place the children in a column without wrapping

LV_FLEX_FLOW_ROW_WRAP: Place the children in a row with wrapping

LV_FLEX_FLOW_COLUMN_WRAP: Place the children in a column with wrapping

LV_FLEX_FLOW_ROW_REVERSE: Place the children in a row without wrapping but in reversed order

LV_FLEX_FLOW_COLUMN_REVERSE: Place the children in a column without wrapping but in reversed order

LV_FLEX_FLOW_ROW_WRAP_REVERSE: Place the children in a row with wrapping but in reversed order

LV_FLEX_FLOW_COLUMN_WRAP_REVERSE: Place the children in a column with wrapping but in reversed order



The possible values are:

LV_FLEX_ALIGN_START: means left on a horizontally and top vertically. (default)

LV_FLEX_ALIGN_END: means right on a horizontally and bottom vertically

LV_FLEX_ALIGN_CENTER: simply center

LV_FLEX_ALIGN_SPACE_EVENLY: items are distributed so that the spacing between any two items (and the space to the edges) is equal. Does not apply to track_cross_place.

LV_FLEX_ALIGN_SPACE_AROUND: items are evenly distributed in the track with equal space around them. Note that visually the spaces aren't equal, since all the items have equal space on both sides. The first item will have one unit of space against the container edge, but two units of space between the next item because that next item has its own spacing that applies. Not applies to track_cross_place.

LV_FLEX_ALIGN_SPACE_BETWEEN: items are evenly distributed in the track: first item is on the start line, last item on the end line. Not applies to track_cross_place.