Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3d83c41905 | |||
| 27300025f3 | |||
| 230dfc5a2b | |||
| f4760c5d3e |
@@ -0,0 +1,144 @@
|
||||
---
|
||||
name: 缓存数据库表重构
|
||||
overview: 重构数据库缓存模块,将 cached 表替换为符合设计规范的 hotime_cache 表,支持 MySQL/SQLite/PostgreSQL,修复并发问题,支持可配置的历史记录功能和自动迁移。
|
||||
todos:
|
||||
- id: update-cache-db
|
||||
content: 重构 cache_db.go:新表、UPSERT、历史记录、自动迁移、问题修复
|
||||
status: completed
|
||||
- id: update-cache-go
|
||||
content: 修改 cache.go:添加 HistorySet 配置传递
|
||||
status: completed
|
||||
- id: update-makecode
|
||||
content: 修改 makecode.go:跳过新表名
|
||||
status: completed
|
||||
---
|
||||
|
||||
# 缓存数据库表重构计划
|
||||
|
||||
## 设计概要
|
||||
|
||||
将原 `cached` 表替换为 `hotime_cache` 表,遵循数据库设计规范,支持:
|
||||
|
||||
- 多数据库:MySQL、SQLite、PostgreSQL
|
||||
- 可配置的历史记录功能(仅日志,无读取接口)
|
||||
- 自动迁移旧 cached 表数据
|
||||
|
||||
## 文件改动清单
|
||||
|
||||
| 文件 | 改动 | 状态 |
|
||||
|
||||
|------|------|------|
|
||||
|
||||
| [cache/cache_db.go](cache/cache_db.go) | 完全重构:新表、UPSERT、历史记录、自动迁移 | 待完成 |
|
||||
|
||||
| [cache/cache.go](cache/cache.go) | 添加 `HistorySet: db.GetBool("history")` 配置传递 | 待完成 |
|
||||
|
||||
| [code/makecode.go](code/makecode.go) | 跳过 hotime_cache 和 hotime_cache_history | 已完成 |
|
||||
|
||||
## 表结构设计
|
||||
|
||||
### 主表 `hotime_cache`
|
||||
|
||||
```sql
|
||||
-- MySQL
|
||||
CREATE TABLE `hotime_cache` (
|
||||
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
|
||||
`key` varchar(64) NOT NULL COMMENT '缓存键',
|
||||
`value` text DEFAULT NULL COMMENT '缓存值',
|
||||
`end_time` datetime DEFAULT NULL COMMENT '过期时间',
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏',
|
||||
`create_time` datetime DEFAULT NULL COMMENT '创建日期',
|
||||
`modify_time` datetime DEFAULT NULL COMMENT '变更时间',
|
||||
PRIMARY KEY (`id`),
|
||||
UNIQUE KEY `uk_key` (`key`),
|
||||
KEY `idx_end_time` (`end_time`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='缓存管理';
|
||||
|
||||
-- SQLite
|
||||
CREATE TABLE "hotime_cache" (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"key" TEXT NOT NULL UNIQUE,
|
||||
"value" TEXT,
|
||||
"end_time" TEXT,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TEXT,
|
||||
"modify_time" TEXT
|
||||
);
|
||||
|
||||
-- PostgreSQL
|
||||
CREATE TABLE "hotime_cache" (
|
||||
"id" SERIAL PRIMARY KEY,
|
||||
"key" VARCHAR(64) NOT NULL UNIQUE,
|
||||
"value" TEXT,
|
||||
"end_time" TIMESTAMP,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TIMESTAMP,
|
||||
"modify_time" TIMESTAMP
|
||||
);
|
||||
CREATE INDEX "idx_hotime_cache_end_time" ON "hotime_cache" ("end_time");
|
||||
```
|
||||
|
||||
### 历史表 `hotime_cache_history`(配置开启时创建,创建后永不自动删除)
|
||||
|
||||
与主表结构相同,但 `id` 改为 `hotime_cache_id` 作为外键关联。
|
||||
|
||||
## 问题修复清单
|
||||
|
||||
| 问题 | 原状态 | 修复方案 |
|
||||
|
||||
|------|--------|----------|
|
||||
|
||||
| key 无索引 | 全表扫描 | 唯一索引 uk_key |
|
||||
|
||||
| 并发竞态 | Update+Insert 可能重复 | 使用 UPSERT 语法 |
|
||||
|
||||
| 时间字段混乱 | time(纳秒) + endtime(秒) | 统一 datetime 格式 |
|
||||
|
||||
| value 长度限制 | varchar(2000) | TEXT 类型 |
|
||||
|
||||
| TimeOut=0 立即过期 | 无默认值 | 默认 24 小时 |
|
||||
|
||||
| get 时删除过期数据 | 每次写操作 | 惰性删除,只返回 nil |
|
||||
|
||||
| 旧表 key 重复 | 无约束 | 迁移时取最后一条 |
|
||||
|
||||
| value 包装冗余 | `{"data": value}` | 直接存储 |
|
||||
|
||||
## 主要代码改动点
|
||||
|
||||
### cache_db.go 改动
|
||||
|
||||
1. 新增常量:表名、默认过期时间 24 小时
|
||||
2. 结构体添加 `HistorySet bool`
|
||||
3. 使用 common 包 `Time2Str(time.Now())` 格式化时间
|
||||
4. `initDbTable()`: 支持三种数据库、自动迁移、创建历史表
|
||||
5. `migrateFromCached()`: 去重迁移(取最后一条)、删除旧表
|
||||
6. `writeHistory()`: 查询数据写入历史表(仅日志)
|
||||
7. `set()`: 使用 UPSERT、调用 writeHistory
|
||||
8. `get()`: 惰性删除
|
||||
9. `Cache()`: TimeOut=0 时用默认值
|
||||
|
||||
### cache.go 改动
|
||||
|
||||
第 268 行添加:`HistorySet: db.GetBool("history")`
|
||||
|
||||
## 配置示例
|
||||
|
||||
```json
|
||||
{
|
||||
"cache": {
|
||||
"db": {
|
||||
"db": true,
|
||||
"session": true,
|
||||
"timeout": 72000,
|
||||
"history": false
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 历史记录逻辑
|
||||
|
||||
- 新增/修改后:查询完整数据,id 改为 hotime_cache_id,插入历史表
|
||||
- 删除:不记录历史
|
||||
- 历史表仅日志记录,无读取接口,人工在数据库操作
|
||||
@@ -0,0 +1,221 @@
|
||||
---
|
||||
name: 缓存表模式配置
|
||||
overview: 为 CacheDb 添加 mode 配置项,支持 "new"(默认,只用新表)和 "compatible"(写新读老)两种模式,并更新配置说明。
|
||||
todos:
|
||||
- id: add-mode-field
|
||||
content: 在 CacheDb 结构体中添加 Mode 字段
|
||||
status: completed
|
||||
- id: modify-init
|
||||
content: 修改 initDbTable,根据 Mode 决定是否迁移删除老表
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-mode-field
|
||||
- id: add-legacy-get
|
||||
content: 添加 getLegacy 方法读取老表数据(unix时间戳格式)
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-mode-field
|
||||
- id: modify-get
|
||||
content: 修改 get 方法,compatible 模式下回退读取老表
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-legacy-get
|
||||
- id: modify-set
|
||||
content: 修改 set 方法,compatible 模式下写新表后删除老表同key记录
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-mode-field
|
||||
- id: modify-delete
|
||||
content: 修改 delete 方法,compatible 模式下同时删除新表和老表
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-mode-field
|
||||
- id: update-cache-init
|
||||
content: 在 cache.go Init 方法中读取 mode 配置
|
||||
status: completed
|
||||
dependencies:
|
||||
- add-mode-field
|
||||
- id: update-config-note
|
||||
content: 在 var.go ConfigNote 中添加 mode 配置说明
|
||||
status: completed
|
||||
- id: add-cache-test
|
||||
content: 在 example/main.go 中添加缓存测试路由
|
||||
status: completed
|
||||
dependencies:
|
||||
- modify-get
|
||||
- modify-set
|
||||
- modify-delete
|
||||
- update-cache-init
|
||||
- id: todo-1769763169689-k7t9twp5t
|
||||
content: |
|
||||
QUICKSTART.md 更新:缓存配置部分需要添加 mode 和 history 配置说明
|
||||
status: pending
|
||||
---
|
||||
|
||||
# 缓存表模式配置实现计划
|
||||
|
||||
## 需求概述
|
||||
|
||||
在 [`cache/cache_db.go`](cache/cache_db.go) 中实现两种缓存表模式:
|
||||
|
||||
- **new**(默认):只使用新的 `hotime_cache` 表,自动迁移老表数据
|
||||
- **compatible**:写入新表,读取时先查新表再查老表,老数据自然过期消亡
|
||||
|
||||
## 实现步骤
|
||||
|
||||
### 1. 修改 CacheDb 结构体
|
||||
|
||||
在 [`cache/cache_db.go`](cache/cache_db.go) 中添加 `Mode` 字段:
|
||||
|
||||
```go
|
||||
type CacheDb struct {
|
||||
TimeOut int64
|
||||
DbSet bool
|
||||
SessionSet bool
|
||||
HistorySet bool
|
||||
Mode string // "new"(默认) 或 "compatible"
|
||||
Db HoTimeDBInterface
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 修改初始化逻辑 initDbTable
|
||||
|
||||
- **new 模式**:创建新表、迁移老表数据,**但不删除老表**(删除交给用户手动操作,更安全)
|
||||
- **compatible 模式**:创建新表,不迁移也不删除老表
|
||||
|
||||
两种模式都不自动删除老表,避免自动删除造成数据丢失风险
|
||||
|
||||
### 3. 修改 get 方法
|
||||
|
||||
- **new 模式**:只从新表读取
|
||||
- **compatible 模式**:先从新表读取,如果没有再从老表读取
|
||||
|
||||
需要新增 `getLegacy` 方法来读取老表数据,该方法需要:
|
||||
|
||||
1. 查询老表数据
|
||||
2. 检查 `endtime`(unix 时间戳)是否过期
|
||||
3. 如果过期:删除该条记录,返回 nil
|
||||
4. 如果未过期:返回数据
|
||||
|
||||
### 4. 修改 set 方法(写新删老)
|
||||
|
||||
- **new 模式**:只写新表(老表保留但不再管理)
|
||||
- **compatible 模式**:写入新表 + 删除老表中相同 key 的记录
|
||||
|
||||
这样可以主动加速老数据消亡,而不是等自然过期。
|
||||
|
||||
### 5. 修改 delete 方法
|
||||
|
||||
- **new 模式**:只删除新表(老表保留但不再管理)
|
||||
- **compatible 模式**:同时删除新表和老表中的 key
|
||||
|
||||
这是必要的,否则删除新表后,下次读取会回退读到老表的数据,造成"删不掉"的问题。
|
||||
|
||||
需要新增 `deleteLegacy` 方法处理老表删除逻辑
|
||||
|
||||
### 6. 更新缓存初始化
|
||||
|
||||
在 [`cache/cache.go`](cache/cache.go) 的 `Init` 方法中读取 `mode` 配置:
|
||||
|
||||
```go
|
||||
that.dbCache = &CacheDb{
|
||||
// ...
|
||||
Mode: db.GetString("mode"), // 读取 mode 配置
|
||||
}
|
||||
```
|
||||
|
||||
### 7. 更新配置说明
|
||||
|
||||
在 [`var.go`](var.go) 的 `ConfigNote` 中添加 `mode` 配置说明:
|
||||
|
||||
```go
|
||||
"db": Map{
|
||||
// ...
|
||||
"mode": "默认new,非必须,new为只使用新表(自动迁移老数据),compatible为兼容模式(写新表读老表,老数据自然过期)",
|
||||
}
|
||||
```
|
||||
|
||||
### 8. 编写测试
|
||||
|
||||
在 [`example/main.go`](example/main.go) 中添加缓存测试路由,测试覆盖:
|
||||
|
||||
**new 模式测试:**
|
||||
|
||||
- 基础读写:set/get/delete 正常工作
|
||||
- 过期测试:设置短过期时间,验证过期后读取返回 nil
|
||||
- 数据迁移:验证老表数据能正确迁移到新表
|
||||
- 老表保留:验证迁移后老表仍存在(不自动删除)
|
||||
|
||||
**compatible 模式测试:**
|
||||
|
||||
- 新表读写:优先从新表读取
|
||||
- 老表回退:新表没有时从老表读取
|
||||
- 过期检测:读取老表过期数据时返回 nil 并删除该记录
|
||||
- 写新删老:写入新表后老表同 key 记录被删除
|
||||
- 删除双表:删除操作同时删除新表和老表记录
|
||||
- 通配删除:`key*` 格式删除测试
|
||||
|
||||
**边界情况测试:**
|
||||
|
||||
- 空值处理:nil 值的 set/get
|
||||
- 不存在的 key 读取
|
||||
- 重复 set 同一个 key
|
||||
- 超时时间参数测试(默认/自定义)
|
||||
|
||||
## 架构图
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Config[配置]
|
||||
ModeNew["mode: new (默认)"]
|
||||
ModeCompat["mode: compatible"]
|
||||
end
|
||||
|
||||
subgraph NewMode[new模式]
|
||||
N1[初始化] --> N2[创建新表]
|
||||
N2 --> N3{老表存在?}
|
||||
N3 -->|是| N4[迁移数据到新表]
|
||||
N4 --> N5[保留老表由人工删除]
|
||||
N3 -->|否| N6[完成]
|
||||
N5 --> N6
|
||||
|
||||
NR[读取] --> NR1[只查询新表]
|
||||
NW[写入] --> NW1[只写入新表]
|
||||
ND[删除] --> ND1[只删除新表记录]
|
||||
end
|
||||
|
||||
subgraph CompatMode[compatible模式]
|
||||
C1[初始化] --> C2[创建新表]
|
||||
C2 --> C3[保留老表]
|
||||
|
||||
CR[读取] --> CR1[查询新表]
|
||||
CR1 -->|未找到| CR2[查询老表]
|
||||
CR2 --> CR3{过期?}
|
||||
CR3 -->|是| CR4[删除老表记录]
|
||||
CR4 --> CR5[返回nil]
|
||||
CR3 -->|否| CR6[返回数据]
|
||||
|
||||
CW[写入] --> CW1[写入新表]
|
||||
CW1 --> CW2[删除老表同key]
|
||||
|
||||
CD[删除] --> CD1[删除新表记录]
|
||||
CD1 --> CD2[删除老表记录]
|
||||
end
|
||||
```
|
||||
|
||||
## 文件修改列表
|
||||
|
||||
| 文件 | 修改内容 |
|
||||
|
||||
|------|----------|
|
||||
|
||||
| [`cache/cache_db.go`](cache/cache_db.go) | 添加 Mode 字段、修改 initDbTable、添加 getLegacy(含过期检测删除)、修改 get、修改 set(写新删老)、修改 delete |
|
||||
|
||||
| [`cache/cache.go`](cache/cache.go) | 读取 mode 配置 |
|
||||
|
||||
| [`var.go`](var.go) | ConfigNote 添加 mode 说明 |
|
||||
|
||||
| [`example/main.go`](example/main.go) | 添加缓存测试路由 |
|
||||
|
||||
| [`example/config/config.json`](example/config/config.json) | 可选:添加 mode 配置示例 |
|
||||
@@ -0,0 +1,103 @@
|
||||
---
|
||||
name: 规范文档创建
|
||||
overview: 创建两个规范文档:一个数据库设计规范文档,一个管理后台配置规范文档(包含admin.json和rule.json的配置说明),并在README.md中添加链接。
|
||||
todos:
|
||||
- id: create-db-doc
|
||||
content: 创建 docs/DatabaseDesign_数据库设计规范.md
|
||||
status: pending
|
||||
- id: create-admin-doc
|
||||
content: 创建 docs/AdminConfig_管理后台配置规范.md
|
||||
status: pending
|
||||
- id: update-readme
|
||||
content: 在 README.md 文档表格中添加两个新文档链接
|
||||
status: pending
|
||||
dependencies:
|
||||
- create-db-doc
|
||||
- create-admin-doc
|
||||
---
|
||||
|
||||
# 创建规范文档
|
||||
|
||||
## 任务概述
|
||||
|
||||
在 `D:\work\hotimev1.5\docs` 目录下创建两个规范文档,并更新 README.md 添加链接。
|
||||
|
||||
---
|
||||
|
||||
## 文档一:数据库设计规范
|
||||
|
||||
**文件**: [docs/DatabaseDesign_数据库设计规范.md](docs/DatabaseDesign_数据库设计规范.md)
|
||||
|
||||
### 内容结构
|
||||
|
||||
| 章节 | 内容 |
|
||||
|
||||
|------|------|
|
||||
|
||||
| 表命名规则 | 不加前缀、可用简称、关联表命名(主表_关联表) |
|
||||
|
||||
| 字段命名规则 | 主键id、外键表名_id、全局唯一性要求、层级字段(parent_id/parent_ids/level) |
|
||||
|
||||
| 注释规则 | select类型格式(`状态:0-正常,1-异常`)、时间用datetime |
|
||||
|
||||
| 必有字段 | state、create_time、modify_time |
|
||||
|
||||
| 示例 | 完整建表SQL示例 |
|
||||
|
||||
---
|
||||
|
||||
## 文档二:管理后台配置规范
|
||||
|
||||
**文件**: [docs/AdminConfig_管理后台配置规范.md](docs/AdminConfig_管理后台配置规范.md)
|
||||
|
||||
### 内容结构
|
||||
|
||||
| 章节 | 内容 |
|
||||
|
||||
|------|------|
|
||||
|
||||
| admin.json 配置 | |
|
||||
|
||||
| - flow配置 | 数据流控制,定义表间权限关系(sql条件、stop标志) |
|
||||
|
||||
| - labelConfig | 操作按钮标签(show/add/delete/edit/info/download) |
|
||||
|
||||
| - label | 菜单/表的显示名称 |
|
||||
|
||||
| - menus配置 | 菜单结构(嵌套menus、icon、table、name、auth) |
|
||||
|
||||
| - auth配置 | 权限数组(show/add/delete/edit/info/download) |
|
||||
|
||||
| - icon配置 | 菜单图标(如Setting) |
|
||||
|
||||
| - table/name配置 | table指定数据表,name用于分组标识 |
|
||||
|
||||
| - stop配置 | 不允许用户修改自身关联数据的表 |
|
||||
|
||||
| rule.json 配置 | |
|
||||
|
||||
| - 字段默认权限 | add/edit/info/list/must/strict/type 各字段含义 |
|
||||
|
||||
| - 内置字段规则 | id、parent_id、create_time、modify_time、password等 |
|
||||
|
||||
| - type类型说明 | select/time/image/file/password/textArea/auth/form等 |
|
||||
|
||||
---
|
||||
|
||||
## 更新 README.md
|
||||
|
||||
在文档表格中添加两个新链接:
|
||||
|
||||
```markdown
|
||||
| [数据库设计规范](docs/DatabaseDesign_数据库设计规范.md) | 表命名、字段命名、注释规则、必有字段 |
|
||||
| [管理后台配置规范](docs/AdminConfig_管理后台配置规范.md) | admin.json、rule.json 配置说明 |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 关键参考文件
|
||||
|
||||
- [`code/config.go`](code/config.go): RuleConfig 默认规则定义
|
||||
- [`code/makecode.go`](code/makecode.go): 外键自动关联逻辑
|
||||
- [`example/config/admin.json`](example/config/admin.json): 完整配置示例
|
||||
- [`example/config/rule.json`](example/config/rule.json): 字段规则示例
|
||||
@@ -22,6 +22,9 @@
|
||||
| [HoTimeDB API 参考](docs/HoTimeDB_API参考.md) | 数据库 API 速查手册 |
|
||||
| [Common 工具类](docs/Common_工具类使用说明.md) | Map/Slice/Obj 类型、类型转换、工具函数 |
|
||||
| [代码生成器](docs/CodeGen_使用说明.md) | 自动 CRUD 代码生成、配置规则 |
|
||||
| [数据库设计规范](docs/DatabaseDesign_数据库设计规范.md) | 表命名、字段命名、时间类型、必有字段规范 |
|
||||
| [代码生成配置规范](docs/CodeConfig_代码生成配置规范.md) | codeConfig、菜单权限、字段规则配置说明 |
|
||||
| [改进规划](docs/ROADMAP_改进规划.md) | 待改进项、设计思考、版本迭代规划 |
|
||||
|
||||
## 安装
|
||||
|
||||
|
||||
Vendored
+330
-3
@@ -1,11 +1,36 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
)
|
||||
|
||||
const debugLogPath = `d:\work\hotimev1.5\.cursor\debug.log`
|
||||
|
||||
// debugLog 写入调试日志
|
||||
func debugLog(hypothesisId, location, message string, data map[string]interface{}) {
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(debugLogPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{
|
||||
"sessionId": "cache-debug",
|
||||
"runId": "test-run",
|
||||
"hypothesisId": hypothesisId,
|
||||
"location": location,
|
||||
"message": message,
|
||||
"data": data,
|
||||
"timestamp": time.Now().UnixMilli(),
|
||||
})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// HoTimeCache 可配置memory,db,redis,默认启用memory,默认优先级为memory>redis>db,memory与数据库缓存设置项一致,
|
||||
// 缓存数据填充会自动反方向反哺,加入memory缓存过期将自动从redis更新,但memory永远不会更新redis,如果是集群建议不要开启memory,配置即启用
|
||||
type HoTimeCache struct {
|
||||
@@ -181,6 +206,299 @@ func (that *HoTimeCache) Cache(key string, data ...interface{}) *Obj {
|
||||
return reData
|
||||
}
|
||||
|
||||
// SessionsGet 批量获取 Session 缓存
|
||||
// 返回 Map,key 为缓存键,value 为缓存值
|
||||
// 优先级:memory > redis > db,低优先级数据会反哺到高优先级缓存
|
||||
func (that *HoTimeCache) SessionsGet(keys []string) Map {
|
||||
if len(keys) == 0 {
|
||||
return Map{}
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:start", "SessionsGet开始", map[string]interface{}{
|
||||
"keys_count": len(keys),
|
||||
"has_memory": that.memoryCache != nil,
|
||||
"has_redis": that.redisCache != nil,
|
||||
"has_db": that.dbCache != nil,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
result := make(Map, len(keys))
|
||||
missingKeys := keys
|
||||
|
||||
// 从 memory 获取
|
||||
if that.memoryCache != nil && that.memoryCache.SessionSet {
|
||||
memResult := that.memoryCache.CachesGet(keys)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:memory", "从Memory获取", map[string]interface{}{
|
||||
"found_count": len(memResult),
|
||||
})
|
||||
// #endregion
|
||||
for k, v := range memResult {
|
||||
result[k] = v
|
||||
}
|
||||
// 计算未命中的 keys
|
||||
missingKeys = make([]string, 0)
|
||||
for _, k := range keys {
|
||||
if _, exists := result[k]; !exists {
|
||||
missingKeys = append(missingKeys, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 从 redis 获取未命中的
|
||||
if len(missingKeys) > 0 && that.redisCache != nil && that.redisCache.SessionSet {
|
||||
redisResult := that.redisCache.CachesGet(missingKeys)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:redis", "从Redis获取", map[string]interface{}{
|
||||
"missing_count": len(missingKeys),
|
||||
"found_count": len(redisResult),
|
||||
})
|
||||
// #endregion
|
||||
// 反哺到 memory
|
||||
if that.memoryCache != nil && that.memoryCache.SessionSet && len(redisResult) > 0 {
|
||||
that.memoryCache.CachesSet(redisResult)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:backfill_redis_to_mem", "Redis数据反哺到Memory", map[string]interface{}{
|
||||
"backfill_count": len(redisResult),
|
||||
})
|
||||
// #endregion
|
||||
}
|
||||
for k, v := range redisResult {
|
||||
result[k] = v
|
||||
}
|
||||
// 更新未命中的 keys
|
||||
newMissing := make([]string, 0)
|
||||
for _, k := range missingKeys {
|
||||
if _, exists := result[k]; !exists {
|
||||
newMissing = append(newMissing, k)
|
||||
}
|
||||
}
|
||||
missingKeys = newMissing
|
||||
}
|
||||
|
||||
// 从 db 获取未命中的
|
||||
if len(missingKeys) > 0 && that.dbCache != nil && that.dbCache.SessionSet {
|
||||
dbResult := that.dbCache.CachesGet(missingKeys)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:db", "从DB获取", map[string]interface{}{
|
||||
"missing_count": len(missingKeys),
|
||||
"found_count": len(dbResult),
|
||||
})
|
||||
// #endregion
|
||||
// 反哺到 memory 和 redis
|
||||
if len(dbResult) > 0 {
|
||||
if that.memoryCache != nil && that.memoryCache.SessionSet {
|
||||
that.memoryCache.CachesSet(dbResult)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:backfill_db_to_mem", "DB数据反哺到Memory", map[string]interface{}{
|
||||
"backfill_count": len(dbResult),
|
||||
})
|
||||
// #endregion
|
||||
}
|
||||
if that.redisCache != nil && that.redisCache.SessionSet {
|
||||
that.redisCache.CachesSet(dbResult)
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:backfill_db_to_redis", "DB数据反哺到Redis", map[string]interface{}{
|
||||
"backfill_count": len(dbResult),
|
||||
})
|
||||
// #endregion
|
||||
}
|
||||
}
|
||||
for k, v := range dbResult {
|
||||
result[k] = v
|
||||
}
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "cache.go:SessionsGet:end", "SessionsGet完成", map[string]interface{}{
|
||||
"total_found": len(result),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// SessionsSet 批量设置 Session 缓存
|
||||
// data: Map,key 为缓存键,value 为缓存值
|
||||
func (that *HoTimeCache) SessionsSet(data Map) {
|
||||
if len(data) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("A", "cache.go:SessionsSet:start", "SessionsSet开始", map[string]interface{}{
|
||||
"data_count": len(data),
|
||||
"has_memory": that.memoryCache != nil,
|
||||
"has_redis": that.redisCache != nil,
|
||||
"has_db": that.dbCache != nil,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if that.memoryCache != nil && that.memoryCache.SessionSet {
|
||||
that.memoryCache.CachesSet(data)
|
||||
// #region agent log
|
||||
debugLog("A", "cache.go:SessionsSet:memory", "写入Memory完成", map[string]interface{}{"count": len(data)})
|
||||
// #endregion
|
||||
}
|
||||
if that.redisCache != nil && that.redisCache.SessionSet {
|
||||
that.redisCache.CachesSet(data)
|
||||
// #region agent log
|
||||
debugLog("A", "cache.go:SessionsSet:redis", "写入Redis完成", map[string]interface{}{"count": len(data)})
|
||||
// #endregion
|
||||
}
|
||||
if that.dbCache != nil && that.dbCache.SessionSet {
|
||||
that.dbCache.CachesSet(data)
|
||||
// #region agent log
|
||||
debugLog("A", "cache.go:SessionsSet:db", "写入DB完成", map[string]interface{}{"count": len(data)})
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("A", "cache.go:SessionsSet:end", "SessionsSet完成", nil)
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// SessionsDelete 批量删除 Session 缓存
|
||||
func (that *HoTimeCache) SessionsDelete(keys []string) {
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("C", "cache.go:SessionsDelete:start", "SessionsDelete开始", map[string]interface{}{
|
||||
"keys_count": len(keys),
|
||||
"has_memory": that.memoryCache != nil,
|
||||
"has_redis": that.redisCache != nil,
|
||||
"has_db": that.dbCache != nil,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if that.memoryCache != nil && that.memoryCache.SessionSet {
|
||||
that.memoryCache.CachesDelete(keys)
|
||||
// #region agent log
|
||||
debugLog("C", "cache.go:SessionsDelete:memory", "从Memory删除完成", map[string]interface{}{"count": len(keys)})
|
||||
// #endregion
|
||||
}
|
||||
if that.redisCache != nil && that.redisCache.SessionSet {
|
||||
that.redisCache.CachesDelete(keys)
|
||||
// #region agent log
|
||||
debugLog("C", "cache.go:SessionsDelete:redis", "从Redis删除完成", map[string]interface{}{"count": len(keys)})
|
||||
// #endregion
|
||||
}
|
||||
if that.dbCache != nil && that.dbCache.SessionSet {
|
||||
that.dbCache.CachesDelete(keys)
|
||||
// #region agent log
|
||||
debugLog("C", "cache.go:SessionsDelete:db", "从DB删除完成", map[string]interface{}{"count": len(keys)})
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLog("C", "cache.go:SessionsDelete:end", "SessionsDelete完成", nil)
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// CachesGet 批量获取普通缓存
|
||||
// 返回 Map,key 为缓存键,value 为缓存值
|
||||
// 优先级:memory > redis > db,低优先级数据会反哺到高优先级缓存
|
||||
func (that *HoTimeCache) CachesGet(keys []string) Map {
|
||||
if len(keys) == 0 {
|
||||
return Map{}
|
||||
}
|
||||
|
||||
result := make(Map, len(keys))
|
||||
missingKeys := keys
|
||||
|
||||
// 从 memory 获取
|
||||
if that.memoryCache != nil {
|
||||
memResult := that.memoryCache.CachesGet(keys)
|
||||
for k, v := range memResult {
|
||||
result[k] = v
|
||||
}
|
||||
// 计算未命中的 keys
|
||||
missingKeys = make([]string, 0)
|
||||
for _, k := range keys {
|
||||
if _, exists := result[k]; !exists {
|
||||
missingKeys = append(missingKeys, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 从 redis 获取未命中的
|
||||
if len(missingKeys) > 0 && that.redisCache != nil {
|
||||
redisResult := that.redisCache.CachesGet(missingKeys)
|
||||
// 反哺到 memory
|
||||
if that.memoryCache != nil && len(redisResult) > 0 {
|
||||
that.memoryCache.CachesSet(redisResult)
|
||||
}
|
||||
for k, v := range redisResult {
|
||||
result[k] = v
|
||||
}
|
||||
// 更新未命中的 keys
|
||||
newMissing := make([]string, 0)
|
||||
for _, k := range missingKeys {
|
||||
if _, exists := result[k]; !exists {
|
||||
newMissing = append(newMissing, k)
|
||||
}
|
||||
}
|
||||
missingKeys = newMissing
|
||||
}
|
||||
|
||||
// 从 db 获取未命中的
|
||||
if len(missingKeys) > 0 && that.dbCache != nil {
|
||||
dbResult := that.dbCache.CachesGet(missingKeys)
|
||||
// 反哺到 memory 和 redis
|
||||
if len(dbResult) > 0 {
|
||||
if that.memoryCache != nil {
|
||||
that.memoryCache.CachesSet(dbResult)
|
||||
}
|
||||
if that.redisCache != nil {
|
||||
that.redisCache.CachesSet(dbResult)
|
||||
}
|
||||
}
|
||||
for k, v := range dbResult {
|
||||
result[k] = v
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// CachesSet 批量设置普通缓存
|
||||
// data: Map,key 为缓存键,value 为缓存值
|
||||
func (that *HoTimeCache) CachesSet(data Map) {
|
||||
if len(data) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
if that.memoryCache != nil {
|
||||
that.memoryCache.CachesSet(data)
|
||||
}
|
||||
if that.redisCache != nil {
|
||||
that.redisCache.CachesSet(data)
|
||||
}
|
||||
if that.dbCache != nil {
|
||||
that.dbCache.CachesSet(data)
|
||||
}
|
||||
}
|
||||
|
||||
// CachesDelete 批量删除普通缓存
|
||||
func (that *HoTimeCache) CachesDelete(keys []string) {
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
if that.memoryCache != nil {
|
||||
that.memoryCache.CachesDelete(keys)
|
||||
}
|
||||
if that.redisCache != nil {
|
||||
that.redisCache.CachesDelete(keys)
|
||||
}
|
||||
if that.dbCache != nil {
|
||||
that.dbCache.CachesDelete(keys)
|
||||
}
|
||||
}
|
||||
|
||||
func (that *HoTimeCache) Init(config Map, hotimeDb HoTimeDBInterface, err ...*Error) {
|
||||
//防止空数据问题
|
||||
if config == nil {
|
||||
@@ -263,11 +581,20 @@ func (that *HoTimeCache) Init(config Map, hotimeDb HoTimeDBInterface, err ...*Er
|
||||
if db.Get("timeout") == nil {
|
||||
db["timeout"] = 60 * 60 * 24 * 30
|
||||
}
|
||||
// mode 默认为 "compatible"(兼容模式,便于老系统平滑升级)
|
||||
if db.Get("mode") == nil {
|
||||
db["mode"] = CacheModeCompatible
|
||||
}
|
||||
that.Config["db"] = db
|
||||
|
||||
that.dbCache = &CacheDb{TimeOut: db.GetCeilInt64("timeout"),
|
||||
DbSet: db.GetBool("db"), SessionSet: db.GetBool("session"),
|
||||
Db: hotimeDb}
|
||||
that.dbCache = &CacheDb{
|
||||
TimeOut: db.GetCeilInt64("timeout"),
|
||||
DbSet: db.GetBool("db"),
|
||||
SessionSet: db.GetBool("session"),
|
||||
HistorySet: db.GetBool("history"),
|
||||
Mode: db.GetString("mode"),
|
||||
Db: hotimeDb,
|
||||
}
|
||||
|
||||
if err[0] != nil {
|
||||
that.dbCache.SetError(err[0])
|
||||
|
||||
Vendored
+601
-81
@@ -1,11 +1,43 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
)
|
||||
|
||||
// debugLogDb 写入调试日志
|
||||
func debugLogDb(hypothesisId, location, message string, data map[string]interface{}) {
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(`d:\work\hotimev1.5\.cursor\debug.log`, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{
|
||||
"sessionId": "cache-db-debug",
|
||||
"runId": "test-run",
|
||||
"hypothesisId": hypothesisId,
|
||||
"location": location,
|
||||
"message": message,
|
||||
"data": data,
|
||||
"timestamp": time.Now().UnixMilli(),
|
||||
})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// 表名常量
|
||||
const (
|
||||
CacheTableName = "hotime_cache"
|
||||
CacheHistoryTableName = "hotime_cache_history"
|
||||
LegacyCacheTableName = "cached" // 老版本缓存表名
|
||||
DefaultCacheTimeout = 24 * 60 * 60 // 默认过期时间 24 小时
|
||||
CacheModeNew = "new" // 新模式:只使用新表
|
||||
CacheModeCompatible = "compatible" // 兼容模式:写新读老
|
||||
)
|
||||
|
||||
type HoTimeDBInterface interface {
|
||||
@@ -24,6 +56,8 @@ type CacheDb struct {
|
||||
TimeOut int64
|
||||
DbSet bool
|
||||
SessionSet bool
|
||||
HistorySet bool // 是否开启历史记录
|
||||
Mode string // 缓存模式:"new"(默认,只用新表) 或 "compatible"(写新读老)
|
||||
Db HoTimeDBInterface
|
||||
*Error
|
||||
ContextBase
|
||||
@@ -31,143 +65,629 @@ type CacheDb struct {
|
||||
}
|
||||
|
||||
func (that *CacheDb) GetError() *Error {
|
||||
|
||||
return that.Error
|
||||
|
||||
}
|
||||
|
||||
func (that *CacheDb) SetError(err *Error) {
|
||||
that.Error = err
|
||||
}
|
||||
|
||||
// getTableName 获取带前缀的表名
|
||||
func (that *CacheDb) getTableName() string {
|
||||
return that.Db.GetPrefix() + CacheTableName
|
||||
}
|
||||
|
||||
// getHistoryTableName 获取带前缀的历史表名
|
||||
func (that *CacheDb) getHistoryTableName() string {
|
||||
return that.Db.GetPrefix() + CacheHistoryTableName
|
||||
}
|
||||
|
||||
// getLegacyTableName 获取带前缀的老版本缓存表名
|
||||
func (that *CacheDb) getLegacyTableName() string {
|
||||
return that.Db.GetPrefix() + LegacyCacheTableName
|
||||
}
|
||||
|
||||
// isCompatibleMode 是否为兼容模式
|
||||
func (that *CacheDb) isCompatibleMode() bool {
|
||||
return that.Mode == CacheModeCompatible
|
||||
}
|
||||
|
||||
// getEffectiveMode 获取有效模式(默认为 new)
|
||||
func (that *CacheDb) getEffectiveMode() string {
|
||||
if that.Mode == CacheModeCompatible {
|
||||
return CacheModeCompatible
|
||||
}
|
||||
return CacheModeNew
|
||||
}
|
||||
|
||||
// initDbTable 初始化数据库表
|
||||
func (that *CacheDb) initDbTable() {
|
||||
if that.isInit {
|
||||
return
|
||||
}
|
||||
if that.Db.GetType() == "mysql" {
|
||||
|
||||
dbNames := that.Db.Query("SELECT DATABASE()")
|
||||
|
||||
if len(dbNames) == 0 {
|
||||
return
|
||||
}
|
||||
dbName := dbNames[0].GetString("DATABASE()")
|
||||
res := that.Db.Query("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='" + dbName + "' AND TABLE_NAME='" + that.Db.GetPrefix() + "cached'")
|
||||
if len(res) != 0 {
|
||||
that.isInit = true
|
||||
return
|
||||
}
|
||||
|
||||
_, e := that.Db.Exec("CREATE TABLE `" + that.Db.GetPrefix() + "cached` ( `id` int(11) unsigned NOT NULL AUTO_INCREMENT, `key` varchar(60) DEFAULT NULL, `value` varchar(2000) DEFAULT NULL, `time` bigint(20) DEFAULT NULL, `endtime` bigint(20) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=198740 DEFAULT CHARSET=utf8")
|
||||
if e.GetError() == nil {
|
||||
that.isInit = true
|
||||
}
|
||||
dbType := that.Db.GetType()
|
||||
tableName := that.getTableName()
|
||||
historyTableName := that.getHistoryTableName()
|
||||
legacyTableName := that.getLegacyTableName()
|
||||
|
||||
// 检查并创建主表
|
||||
if !that.tableExists(tableName) {
|
||||
that.createMainTable(dbType, tableName)
|
||||
}
|
||||
|
||||
if that.Db.GetType() == "sqlite" {
|
||||
res := that.Db.Query(`select * from sqlite_master where type = 'table' and name = '` + that.Db.GetPrefix() + `cached'`)
|
||||
|
||||
if len(res) != 0 {
|
||||
that.isInit = true
|
||||
return
|
||||
}
|
||||
_, e := that.Db.Exec(`CREATE TABLE "` + that.Db.GetPrefix() + `cached" (
|
||||
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
|
||||
"key" TEXT(60),
|
||||
"value" TEXT(2000),
|
||||
"time" integer,
|
||||
"endtime" integer
|
||||
);`)
|
||||
if e.GetError() == nil {
|
||||
that.isInit = true
|
||||
// 根据模式处理老表
|
||||
// new 模式:迁移老表数据到新表(不删除老表,由人工删除)
|
||||
// compatible 模式:不迁移,老表继续使用
|
||||
if that.getEffectiveMode() == CacheModeNew {
|
||||
if that.tableExists(legacyTableName) {
|
||||
that.migrateFromCached(dbType, legacyTableName, tableName)
|
||||
}
|
||||
}
|
||||
// compatible 模式不做任何处理,老表保留供读取
|
||||
|
||||
// 检查并创建历史表(开启历史记录时)
|
||||
if that.HistorySet && !that.tableExists(historyTableName) {
|
||||
that.createHistoryTable(dbType, historyTableName)
|
||||
}
|
||||
|
||||
that.isInit = true
|
||||
}
|
||||
|
||||
// 获取Cache键只能为string类型
|
||||
func (that *CacheDb) get(key string) interface{} {
|
||||
// tableExists 检查表是否存在
|
||||
func (that *CacheDb) tableExists(tableName string) bool {
|
||||
dbType := that.Db.GetType()
|
||||
|
||||
cached := that.Db.Get("cached", "*", Map{"key": key})
|
||||
switch dbType {
|
||||
case "mysql":
|
||||
dbNames := that.Db.Query("SELECT DATABASE()")
|
||||
if len(dbNames) == 0 {
|
||||
return false
|
||||
}
|
||||
dbName := dbNames[0].GetString("DATABASE()")
|
||||
res := that.Db.Query("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='" + dbName + "' AND TABLE_NAME='" + tableName + "'")
|
||||
return len(res) != 0
|
||||
|
||||
case "sqlite":
|
||||
res := that.Db.Query(`SELECT name FROM sqlite_master WHERE type='table' AND name='` + tableName + `'`)
|
||||
return len(res) != 0
|
||||
|
||||
case "postgres":
|
||||
res := that.Db.Query(`SELECT tablename FROM pg_tables WHERE schemaname='public' AND tablename='` + tableName + `'`)
|
||||
return len(res) != 0
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// createMainTable 创建主表
|
||||
func (that *CacheDb) createMainTable(dbType, tableName string) {
|
||||
var createSQL string
|
||||
|
||||
switch dbType {
|
||||
case "mysql":
|
||||
createSQL = "CREATE TABLE `" + tableName + "` (" +
|
||||
"`id` int(11) unsigned NOT NULL AUTO_INCREMENT," +
|
||||
"`key` varchar(64) NOT NULL COMMENT '缓存键'," +
|
||||
"`value` text DEFAULT NULL COMMENT '缓存值'," +
|
||||
"`end_time` datetime DEFAULT NULL COMMENT '过期时间'," +
|
||||
"`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏'," +
|
||||
"`create_time` datetime DEFAULT NULL COMMENT '创建日期'," +
|
||||
"`modify_time` datetime DEFAULT NULL COMMENT '变更时间'," +
|
||||
"PRIMARY KEY (`id`)," +
|
||||
"UNIQUE KEY `uk_key` (`key`)," +
|
||||
"KEY `idx_end_time` (`end_time`)" +
|
||||
") ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='缓存管理'"
|
||||
|
||||
case "sqlite":
|
||||
createSQL = `CREATE TABLE "` + tableName + `" (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"key" TEXT NOT NULL UNIQUE,
|
||||
"value" TEXT,
|
||||
"end_time" TEXT,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TEXT,
|
||||
"modify_time" TEXT
|
||||
)`
|
||||
|
||||
case "postgres":
|
||||
createSQL = `CREATE TABLE "` + tableName + `" (
|
||||
"id" SERIAL PRIMARY KEY,
|
||||
"key" VARCHAR(64) NOT NULL UNIQUE,
|
||||
"value" TEXT,
|
||||
"end_time" TIMESTAMP,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TIMESTAMP,
|
||||
"modify_time" TIMESTAMP
|
||||
)`
|
||||
that.Db.Exec(createSQL)
|
||||
// 创建索引
|
||||
that.Db.Exec(`CREATE INDEX "idx_` + tableName + `_end_time" ON "` + tableName + `" ("end_time")`)
|
||||
return
|
||||
}
|
||||
|
||||
that.Db.Exec(createSQL)
|
||||
}
|
||||
|
||||
// createHistoryTable 创建历史表
|
||||
func (that *CacheDb) createHistoryTable(dbType, tableName string) {
|
||||
var createSQL string
|
||||
|
||||
switch dbType {
|
||||
case "mysql":
|
||||
createSQL = "CREATE TABLE `" + tableName + "` (" +
|
||||
"`id` int(11) unsigned NOT NULL AUTO_INCREMENT," +
|
||||
"`hotime_cache_id` int(11) unsigned DEFAULT NULL COMMENT '缓存ID'," +
|
||||
"`key` varchar(64) DEFAULT NULL COMMENT '缓存键'," +
|
||||
"`value` text DEFAULT NULL COMMENT '缓存值'," +
|
||||
"`end_time` datetime DEFAULT NULL COMMENT '过期时间'," +
|
||||
"`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏'," +
|
||||
"`create_time` datetime DEFAULT NULL COMMENT '创建日期'," +
|
||||
"`modify_time` datetime DEFAULT NULL COMMENT '变更时间'," +
|
||||
"PRIMARY KEY (`id`)," +
|
||||
"KEY `idx_hotime_cache_id` (`hotime_cache_id`)" +
|
||||
") ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='缓存历史'"
|
||||
|
||||
case "sqlite":
|
||||
createSQL = `CREATE TABLE "` + tableName + `" (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"hotime_cache_id" INTEGER,
|
||||
"key" TEXT,
|
||||
"value" TEXT,
|
||||
"end_time" TEXT,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TEXT,
|
||||
"modify_time" TEXT
|
||||
)`
|
||||
|
||||
case "postgres":
|
||||
createSQL = `CREATE TABLE "` + tableName + `" (
|
||||
"id" SERIAL PRIMARY KEY,
|
||||
"hotime_cache_id" INTEGER,
|
||||
"key" VARCHAR(64),
|
||||
"value" TEXT,
|
||||
"end_time" TIMESTAMP,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TIMESTAMP,
|
||||
"modify_time" TIMESTAMP
|
||||
)`
|
||||
that.Db.Exec(createSQL)
|
||||
// 创建索引
|
||||
that.Db.Exec(`CREATE INDEX "idx_` + tableName + `_cache_id" ON "` + tableName + `" ("hotime_cache_id")`)
|
||||
return
|
||||
}
|
||||
|
||||
that.Db.Exec(createSQL)
|
||||
}
|
||||
|
||||
// migrateFromCached 从旧 cached 表迁移数据(不删除老表,由人工删除)
|
||||
func (that *CacheDb) migrateFromCached(dbType, oldTableName, newTableName string) {
|
||||
var migrateSQL string
|
||||
|
||||
switch dbType {
|
||||
case "mysql":
|
||||
// 去重迁移:取每个 key 的最后一条记录(id 最大)
|
||||
// 使用 INSERT IGNORE 避免重复 key 冲突
|
||||
migrateSQL = "INSERT IGNORE INTO `" + newTableName + "` (`key`, `value`, `end_time`, `state`, `create_time`, `modify_time`) " +
|
||||
"SELECT c.`key`, c.`value`, FROM_UNIXTIME(c.`endtime`), 0, " +
|
||||
"FROM_UNIXTIME(c.`time` / 1000000000), FROM_UNIXTIME(c.`time` / 1000000000) " +
|
||||
"FROM `" + oldTableName + "` c " +
|
||||
"INNER JOIN (SELECT `key`, MAX(id) as max_id FROM `" + oldTableName + "` GROUP BY `key`) m " +
|
||||
"ON c.id = m.max_id"
|
||||
|
||||
case "sqlite":
|
||||
migrateSQL = `INSERT OR IGNORE INTO "` + newTableName + `" ("key", "value", "end_time", "state", "create_time", "modify_time") ` +
|
||||
`SELECT c."key", c."value", datetime(c."endtime", 'unixepoch'), 0, ` +
|
||||
`datetime(c."time" / 1000000000, 'unixepoch'), datetime(c."time" / 1000000000, 'unixepoch') ` +
|
||||
`FROM "` + oldTableName + `" c ` +
|
||||
`INNER JOIN (SELECT "key", MAX(id) as max_id FROM "` + oldTableName + `" GROUP BY "key") m ` +
|
||||
`ON c.id = m.max_id`
|
||||
|
||||
case "postgres":
|
||||
migrateSQL = `INSERT INTO "` + newTableName + `" ("key", "value", "end_time", "state", "create_time", "modify_time") ` +
|
||||
`SELECT c."key", c."value", to_timestamp(c."endtime"), 0, ` +
|
||||
`to_timestamp(c."time" / 1000000000), to_timestamp(c."time" / 1000000000) ` +
|
||||
`FROM "` + oldTableName + `" c ` +
|
||||
`INNER JOIN (SELECT "key", MAX(id) as max_id FROM "` + oldTableName + `" GROUP BY "key") m ` +
|
||||
`ON c.id = m.max_id ` +
|
||||
`ON CONFLICT ("key") DO NOTHING`
|
||||
}
|
||||
|
||||
// 执行迁移,不删除老表(由人工确认后删除,更安全)
|
||||
that.Db.Exec(migrateSQL)
|
||||
}
|
||||
|
||||
// writeHistory 写入历史记录
|
||||
func (that *CacheDb) writeHistory(key string) {
|
||||
if !that.HistorySet {
|
||||
return
|
||||
}
|
||||
|
||||
tableName := that.getTableName()
|
||||
historyTableName := that.getHistoryTableName()
|
||||
|
||||
// 查询当前数据
|
||||
cached := that.Db.Get(tableName, "*", Map{"key": key})
|
||||
if cached == nil {
|
||||
return
|
||||
}
|
||||
|
||||
// 构建历史记录数据
|
||||
historyData := Map{
|
||||
"hotime_cache_id": cached.GetInt64("id"),
|
||||
"key": cached.GetString("key"),
|
||||
"value": cached.GetString("value"),
|
||||
"end_time": cached.GetString("end_time"),
|
||||
"state": cached.GetInt("state"),
|
||||
"create_time": cached.GetString("create_time"),
|
||||
"modify_time": cached.GetString("modify_time"),
|
||||
}
|
||||
|
||||
// 插入历史表
|
||||
that.Db.Insert(historyTableName, historyData)
|
||||
}
|
||||
|
||||
// getLegacy 从老表获取缓存(兼容模式使用)
|
||||
// 老表结构:key, value, endtime(unix秒), time(纳秒时间戳)
|
||||
func (that *CacheDb) getLegacy(key string) interface{} {
|
||||
legacyTableName := that.getLegacyTableName()
|
||||
|
||||
// 检查老表是否存在
|
||||
if !that.tableExists(legacyTableName) {
|
||||
return nil
|
||||
}
|
||||
|
||||
cached := that.Db.Get(legacyTableName, "*", Map{"key": key})
|
||||
if cached == nil {
|
||||
return nil
|
||||
}
|
||||
//data:=cacheMap[key];
|
||||
if cached.GetInt64("endtime") <= time.Now().Unix() {
|
||||
|
||||
that.Db.Delete("cached", Map{"id": cached.GetString("id")})
|
||||
// 检查过期时间(老表使用 unix 时间戳)
|
||||
endTime := cached.GetInt64("endtime")
|
||||
nowUnix := time.Now().Unix()
|
||||
if endTime > 0 && endTime <= nowUnix {
|
||||
// 已过期,删除该条记录
|
||||
that.deleteLegacy(key)
|
||||
return nil
|
||||
}
|
||||
|
||||
data := Map{}
|
||||
data.JsonToMap(cached.GetString("value"))
|
||||
|
||||
return data.Get("data")
|
||||
}
|
||||
|
||||
// key value ,时间为时间戳
|
||||
func (that *CacheDb) set(key string, value interface{}, tim int64) {
|
||||
|
||||
bte, _ := json.Marshal(Map{"data": value})
|
||||
|
||||
num := that.Db.Update("cached", Map{"value": string(bte), "time": time.Now().UnixNano(), "endtime": tim}, Map{"key": key})
|
||||
if num == int64(0) {
|
||||
that.Db.Insert("cached", Map{"value": string(bte), "time": time.Now().UnixNano(), "endtime": tim, "key": key})
|
||||
// 解析 value(老表 value 格式可能是 {"data": xxx} 或直接值)
|
||||
valueStr := cached.GetString("value")
|
||||
if valueStr == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
//随机执行删除命令
|
||||
if Rand(1000) > 950 {
|
||||
that.Db.Delete("cached", Map{"endtime[<]": time.Now().Unix()})
|
||||
var data interface{}
|
||||
err := json.Unmarshal([]byte(valueStr), &data)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
// 兼容老版本 {"data": xxx} 包装格式
|
||||
if dataMap, ok := data.(map[string]interface{}); ok {
|
||||
if innerData, exists := dataMap["data"]; exists {
|
||||
return innerData
|
||||
}
|
||||
}
|
||||
|
||||
return data
|
||||
}
|
||||
|
||||
func (that *CacheDb) delete(key string) {
|
||||
// deleteLegacy 从老表删除缓存(兼容模式使用)
|
||||
func (that *CacheDb) deleteLegacy(key string) {
|
||||
legacyTableName := that.getLegacyTableName()
|
||||
|
||||
// 检查老表是否存在
|
||||
if !that.tableExists(legacyTableName) {
|
||||
return
|
||||
}
|
||||
|
||||
del := strings.Index(key, "*")
|
||||
//如果通配删除
|
||||
// 如果通配删除
|
||||
if del != -1 {
|
||||
key = Substr(key, 0, del)
|
||||
that.Db.Delete("cached", Map{"key": key + "%"})
|
||||
|
||||
keyPrefix := Substr(key, 0, del)
|
||||
that.Db.Delete(legacyTableName, Map{"key[~]": keyPrefix + "%"})
|
||||
} else {
|
||||
that.Db.Delete("cached", Map{"key": key})
|
||||
that.Db.Delete(legacyTableName, Map{"key": key})
|
||||
}
|
||||
}
|
||||
|
||||
func (that *CacheDb) Cache(key string, data ...interface{}) *Obj {
|
||||
// get 获取缓存
|
||||
func (that *CacheDb) get(key string) interface{} {
|
||||
tableName := that.getTableName()
|
||||
cached := that.Db.Get(tableName, "*", Map{"key": key})
|
||||
|
||||
if cached != nil {
|
||||
// 使用字符串比较判断过期(ISO 格式天然支持)
|
||||
endTime := cached.GetString("end_time")
|
||||
nowTime := Time2Str(time.Now())
|
||||
if endTime != "" && endTime <= nowTime {
|
||||
// 惰性删除:过期只返回 nil,不立即删除
|
||||
// 依赖随机清理批量删除过期数据
|
||||
// 继续检查老表(如果是兼容模式)
|
||||
} else {
|
||||
// 直接解析 value,不再需要 {"data": value} 包装
|
||||
valueStr := cached.GetString("value")
|
||||
if valueStr != "" {
|
||||
var data interface{}
|
||||
err := json.Unmarshal([]byte(valueStr), &data)
|
||||
if err == nil {
|
||||
return data
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 兼容模式:新表没有数据时,回退读取老表
|
||||
if that.isCompatibleMode() {
|
||||
return that.getLegacy(key)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// set 设置缓存
|
||||
func (that *CacheDb) set(key string, value interface{}, endTime time.Time) {
|
||||
// 直接序列化 value,不再包装
|
||||
bte, _ := json.Marshal(value)
|
||||
nowTime := Time2Str(time.Now())
|
||||
endTimeStr := Time2Str(endTime)
|
||||
|
||||
dbType := that.Db.GetType()
|
||||
tableName := that.getTableName()
|
||||
|
||||
// 使用 UPSERT 语法解决并发问题
|
||||
switch dbType {
|
||||
case "mysql":
|
||||
upsertSQL := "INSERT INTO `" + tableName + "` (`key`, `value`, `end_time`, `state`, `create_time`, `modify_time`) " +
|
||||
"VALUES (?, ?, ?, 0, ?, ?) " +
|
||||
"ON DUPLICATE KEY UPDATE `value`=VALUES(`value`), `end_time`=VALUES(`end_time`), `modify_time`=VALUES(`modify_time`)"
|
||||
that.Db.Exec(upsertSQL, key, string(bte), endTimeStr, nowTime, nowTime)
|
||||
|
||||
case "sqlite":
|
||||
// SQLite: INSERT OR REPLACE 会删除后插入,所以用 UPSERT 语法
|
||||
upsertSQL := `INSERT INTO "` + tableName + `" ("key", "value", "end_time", "state", "create_time", "modify_time") ` +
|
||||
`VALUES (?, ?, ?, 0, ?, ?) ` +
|
||||
`ON CONFLICT("key") DO UPDATE SET "value"=excluded."value", "end_time"=excluded."end_time", "modify_time"=excluded."modify_time"`
|
||||
that.Db.Exec(upsertSQL, key, string(bte), endTimeStr, nowTime, nowTime)
|
||||
|
||||
case "postgres":
|
||||
upsertSQL := `INSERT INTO "` + tableName + `" ("key", "value", "end_time", "state", "create_time", "modify_time") ` +
|
||||
`VALUES ($1, $2, $3, 0, $4, $5) ` +
|
||||
`ON CONFLICT ("key") DO UPDATE SET "value"=EXCLUDED."value", "end_time"=EXCLUDED."end_time", "modify_time"=EXCLUDED."modify_time"`
|
||||
that.Db.Exec(upsertSQL, key, string(bte), endTimeStr, nowTime, nowTime)
|
||||
|
||||
default:
|
||||
// 兼容其他数据库:使用 Update + Insert
|
||||
num := that.Db.Update(tableName, Map{
|
||||
"value": string(bte),
|
||||
"end_time": endTimeStr,
|
||||
"modify_time": nowTime,
|
||||
}, Map{"key": key})
|
||||
if num == 0 {
|
||||
that.Db.Insert(tableName, Map{
|
||||
"key": key,
|
||||
"value": string(bte),
|
||||
"end_time": endTimeStr,
|
||||
"state": 0,
|
||||
"create_time": nowTime,
|
||||
"modify_time": nowTime,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 写入历史记录
|
||||
that.writeHistory(key)
|
||||
|
||||
// 兼容模式:写新表后删除老表同 key 记录(加速老数据消亡)
|
||||
if that.isCompatibleMode() {
|
||||
that.deleteLegacy(key)
|
||||
}
|
||||
|
||||
// 随机执行删除过期数据命令(5% 概率)
|
||||
if Rand(1000) > 950 {
|
||||
nowTimeStr := Time2Str(time.Now())
|
||||
that.Db.Delete(tableName, Map{"end_time[<]": nowTimeStr})
|
||||
}
|
||||
}
|
||||
|
||||
// delete 删除缓存
|
||||
func (that *CacheDb) delete(key string) {
|
||||
tableName := that.getTableName()
|
||||
del := strings.Index(key, "*")
|
||||
// 如果通配删除
|
||||
if del != -1 {
|
||||
keyPrefix := Substr(key, 0, del)
|
||||
that.Db.Delete(tableName, Map{"key[~]": keyPrefix + "%"})
|
||||
} else {
|
||||
that.Db.Delete(tableName, Map{"key": key})
|
||||
}
|
||||
|
||||
// 兼容模式:同时删除老表中的 key(避免回退读到老数据)
|
||||
if that.isCompatibleMode() {
|
||||
that.deleteLegacy(key)
|
||||
}
|
||||
}
|
||||
|
||||
// Cache 缓存操作入口
|
||||
// 用法:
|
||||
// - Cache(key) - 获取缓存
|
||||
// - Cache(key, value) - 设置缓存(使用默认过期时间)
|
||||
// - Cache(key, value, timeout) - 设置缓存(指定过期时间,单位:秒)
|
||||
// - Cache(key, nil) - 删除缓存
|
||||
func (that *CacheDb) Cache(key string, data ...interface{}) *Obj {
|
||||
that.initDbTable()
|
||||
|
||||
// 获取缓存
|
||||
if len(data) == 0 {
|
||||
return &Obj{Data: that.get(key)}
|
||||
}
|
||||
tim := time.Now().Unix()
|
||||
|
||||
// 删除缓存
|
||||
if len(data) == 1 && data[0] == nil {
|
||||
that.delete(key)
|
||||
return &Obj{Data: nil}
|
||||
}
|
||||
|
||||
// 计算过期时间
|
||||
var timeout int64
|
||||
if len(data) == 1 {
|
||||
if that.TimeOut == 0 {
|
||||
//that.Time = Config.GetInt64("cacheLongTime")
|
||||
// 使用配置的 TimeOut,如果为 0 则使用默认值
|
||||
timeout = that.TimeOut
|
||||
if timeout == 0 {
|
||||
timeout = DefaultCacheTimeout
|
||||
}
|
||||
tim += that.TimeOut
|
||||
}
|
||||
if len(data) == 2 {
|
||||
that.SetError(nil)
|
||||
tempt := ObjToInt64(data[1], that.Error)
|
||||
|
||||
if tempt > tim {
|
||||
tim = tempt
|
||||
} else if that.GetError() == nil {
|
||||
|
||||
tim = tim + tempt
|
||||
} else if len(data) >= 2 {
|
||||
// 使用指定的超时时间
|
||||
var err Error
|
||||
tempTimeout := ObjToInt64(data[1], &err)
|
||||
if err.GetError() == nil && tempTimeout > 0 {
|
||||
timeout = tempTimeout
|
||||
} else {
|
||||
timeout = that.TimeOut
|
||||
if timeout == 0 {
|
||||
timeout = DefaultCacheTimeout
|
||||
}
|
||||
}
|
||||
}
|
||||
that.set(key, data[0], tim)
|
||||
|
||||
endTime := time.Now().Add(time.Duration(timeout) * time.Second)
|
||||
that.set(key, data[0], endTime)
|
||||
return &Obj{Data: nil}
|
||||
}
|
||||
|
||||
// CachesGet 批量获取缓存(使用 IN 查询优化)
|
||||
// 返回 Map,key 为缓存键,value 为缓存值(不存在或过期的 key 不包含在结果中)
|
||||
func (that *CacheDb) CachesGet(keys []string) Map {
|
||||
that.initDbTable()
|
||||
result := make(Map, len(keys))
|
||||
if len(keys) == 0 {
|
||||
return result
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLogDb("E", "cache_db.go:CachesGet:start", "CacheDb.CachesGet开始", map[string]interface{}{
|
||||
"keys_count": len(keys),
|
||||
"keys": keys,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
tableName := that.getTableName()
|
||||
nowTime := Time2Str(time.Now())
|
||||
|
||||
// 使用 IN 查询批量获取
|
||||
cachedList := that.Db.Select(tableName, "*", Map{
|
||||
"key": keys,
|
||||
"end_time[>]": nowTime,
|
||||
})
|
||||
|
||||
// #region agent log
|
||||
debugLogDb("E", "cache_db.go:CachesGet:afterSelect", "DB Select完成", map[string]interface{}{
|
||||
"table": tableName,
|
||||
"now_time": nowTime,
|
||||
"found_rows": len(cachedList),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
for _, cached := range cachedList {
|
||||
valueStr := cached.GetString("value")
|
||||
if valueStr != "" {
|
||||
var data interface{}
|
||||
err := json.Unmarshal([]byte(valueStr), &data)
|
||||
if err == nil {
|
||||
result[cached.GetString("key")] = data
|
||||
} else {
|
||||
// #region agent log
|
||||
debugLogDb("E", "cache_db.go:CachesGet:unmarshalError", "JSON解析失败", map[string]interface{}{
|
||||
"key": cached.GetString("key"),
|
||||
"error": err.Error(),
|
||||
})
|
||||
// #endregion
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 兼容模式:新表没有的 key,回退读取老表
|
||||
if that.isCompatibleMode() {
|
||||
// #region agent log
|
||||
debugLogDb("E", "cache_db.go:CachesGet:compatMode", "兼容模式检查老表", nil)
|
||||
// #endregion
|
||||
for _, key := range keys {
|
||||
if _, exists := result[key]; !exists {
|
||||
legacyData := that.getLegacy(key)
|
||||
if legacyData != nil {
|
||||
result[key] = legacyData
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLogDb("E", "cache_db.go:CachesGet:end", "CacheDb.CachesGet完成", map[string]interface{}{
|
||||
"result_count": len(result),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// CachesSet 批量设置缓存
|
||||
// data: Map,key 为缓存键,value 为缓存值
|
||||
// timeout: 可选过期时间(秒),不传则使用默认超时时间
|
||||
func (that *CacheDb) CachesSet(data Map, timeout ...int64) {
|
||||
that.initDbTable()
|
||||
if len(data) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLogDb("A", "cache_db.go:CachesSet:start", "CacheDb.CachesSet开始", map[string]interface{}{
|
||||
"data_count": len(data),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
// 计算过期时间
|
||||
var tim int64
|
||||
if len(timeout) > 0 && timeout[0] > 0 {
|
||||
tim = timeout[0]
|
||||
} else {
|
||||
tim = that.TimeOut
|
||||
if tim == 0 {
|
||||
tim = DefaultCacheTimeout
|
||||
}
|
||||
}
|
||||
endTime := time.Now().Add(time.Duration(tim) * time.Second)
|
||||
|
||||
// 逐个设置(保持事务一致性和历史记录)
|
||||
for key, value := range data {
|
||||
that.set(key, value, endTime)
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
debugLogDb("A", "cache_db.go:CachesSet:end", "CacheDb.CachesSet完成", map[string]interface{}{
|
||||
"data_count": len(data),
|
||||
"end_time": Time2Str(endTime),
|
||||
})
|
||||
// #endregion
|
||||
}
|
||||
|
||||
// CachesDelete 批量删除缓存
|
||||
func (that *CacheDb) CachesDelete(keys []string) {
|
||||
that.initDbTable()
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
tableName := that.getTableName()
|
||||
|
||||
// 使用 IN 条件批量删除
|
||||
that.Db.Delete(tableName, Map{"key": keys})
|
||||
|
||||
// 兼容模式:同时删除老表中的 keys
|
||||
if that.isCompatibleMode() {
|
||||
legacyTableName := that.getLegacyTableName()
|
||||
if that.tableExists(legacyTableName) {
|
||||
that.Db.Delete(legacyTableName, Map{"key": keys})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Vendored
+39
@@ -113,3 +113,42 @@ func (c *CacheMemory) Cache(key string, data ...interface{}) *Obj {
|
||||
c.set(key, data[0], expireAt)
|
||||
return nil
|
||||
}
|
||||
|
||||
// CachesGet 批量获取缓存
|
||||
// 返回 Map,key 为缓存键,value 为缓存值(不存在或过期的 key 不包含在结果中)
|
||||
func (c *CacheMemory) CachesGet(keys []string) Map {
|
||||
result := make(Map, len(keys))
|
||||
for _, key := range keys {
|
||||
obj := c.get(key)
|
||||
if obj != nil && obj.Data != nil {
|
||||
result[key] = obj.Data
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// CachesSet 批量设置缓存
|
||||
// data: Map,key 为缓存键,value 为缓存值
|
||||
// timeout: 可选过期时间(秒),不传则使用默认超时时间
|
||||
func (c *CacheMemory) CachesSet(data Map, timeout ...int64) {
|
||||
now := time.Now().Unix()
|
||||
expireAt := now + c.TimeOut
|
||||
if len(timeout) > 0 && timeout[0] > 0 {
|
||||
if timeout[0] > now {
|
||||
expireAt = timeout[0]
|
||||
} else {
|
||||
expireAt = now + timeout[0]
|
||||
}
|
||||
}
|
||||
|
||||
for key, value := range data {
|
||||
c.set(key, value, expireAt)
|
||||
}
|
||||
}
|
||||
|
||||
// CachesDelete 批量删除缓存
|
||||
func (c *CacheMemory) CachesDelete(keys []string) {
|
||||
for _, key := range keys {
|
||||
c.delete(key)
|
||||
}
|
||||
}
|
||||
|
||||
Vendored
+96
@@ -188,3 +188,99 @@ func (that *CacheRedis) Cache(key string, data ...interface{}) *Obj {
|
||||
|
||||
return reData
|
||||
}
|
||||
|
||||
// CachesGet 批量获取缓存(使用 Redis MGET 命令优化)
|
||||
// 返回 Map,key 为缓存键,value 为缓存值(不存在的 key 不包含在结果中)
|
||||
func (that *CacheRedis) CachesGet(keys []string) Map {
|
||||
result := make(Map, len(keys))
|
||||
if len(keys) == 0 {
|
||||
return result
|
||||
}
|
||||
|
||||
conn := that.getConn()
|
||||
if conn == nil {
|
||||
return result
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
// 构建 MGET 参数
|
||||
args := make([]interface{}, len(keys))
|
||||
for i, key := range keys {
|
||||
args[i] = key
|
||||
}
|
||||
|
||||
values, err := redis.Strings(conn.Do("MGET", args...))
|
||||
if err != nil {
|
||||
if !strings.Contains(err.Error(), "nil returned") {
|
||||
that.Error.SetError(err)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// 将结果映射回 Map
|
||||
for i, value := range values {
|
||||
if value != "" {
|
||||
result[keys[i]] = value
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// CachesSet 批量设置缓存(使用 Redis pipeline 优化)
|
||||
// data: Map,key 为缓存键,value 为缓存值
|
||||
// timeout: 可选过期时间(秒),不传则使用默认超时时间
|
||||
func (that *CacheRedis) CachesSet(data Map, timeout ...int64) {
|
||||
if len(data) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
conn := that.getConn()
|
||||
if conn == nil {
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
tim := that.TimeOut
|
||||
if len(timeout) > 0 && timeout[0] > 0 {
|
||||
if timeout[0] > tim {
|
||||
tim = timeout[0]
|
||||
} else {
|
||||
tim = tim + timeout[0]
|
||||
}
|
||||
}
|
||||
|
||||
// 使用 pipeline 批量设置
|
||||
conn.Send("MULTI")
|
||||
for key, value := range data {
|
||||
conn.Send("SET", key, ObjToStr(value), "EX", ObjToStr(tim))
|
||||
}
|
||||
_, err := conn.Do("EXEC")
|
||||
if err != nil {
|
||||
that.Error.SetError(err)
|
||||
}
|
||||
}
|
||||
|
||||
// CachesDelete 批量删除缓存(使用 Redis DEL 命令批量删除)
|
||||
func (that *CacheRedis) CachesDelete(keys []string) {
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
conn := that.getConn()
|
||||
if conn == nil {
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
// 构建 DEL 参数
|
||||
args := make([]interface{}, len(keys))
|
||||
for i, key := range keys {
|
||||
args[i] = key
|
||||
}
|
||||
|
||||
_, err := conn.Do("DEL", args...)
|
||||
if err != nil {
|
||||
that.Error.SetError(err)
|
||||
}
|
||||
}
|
||||
|
||||
Vendored
+4
@@ -11,6 +11,10 @@ type CacheIns interface {
|
||||
GetError() *Error
|
||||
SetError(err *Error)
|
||||
Cache(key string, data ...interface{}) *Obj
|
||||
// 批量操作
|
||||
CachesGet(keys []string) Map // 批量获取
|
||||
CachesSet(data Map, timeout ...int64) // 批量设置
|
||||
CachesDelete(keys []string) // 批量删除
|
||||
}
|
||||
|
||||
// 单条缓存数据
|
||||
|
||||
+3
-3
@@ -120,9 +120,9 @@ func (that *MakeCode) Db2JSON(db *db.HoTimeDB, config Map) {
|
||||
}
|
||||
//idSlice=append(idSlice,nowTables)
|
||||
for _, v := range nowTables {
|
||||
if v.GetString("name") == "cached" {
|
||||
continue
|
||||
}
|
||||
// if v.GetString("name") == "cached" {
|
||||
// continue
|
||||
// }
|
||||
if that.TableConfig.GetMap(v.GetString("name")) == nil {
|
||||
if v.GetString("label") == "" {
|
||||
v["label"] = v.GetString("name")
|
||||
|
||||
@@ -0,0 +1,757 @@
|
||||
# 代码生成配置规范
|
||||
|
||||
本文档详细说明 HoTime 框架代码生成器的配置体系,包括 `config.json` 中的 `codeConfig`、菜单权限配置和字段规则配置。
|
||||
|
||||
---
|
||||
|
||||
## 配置体系概览
|
||||
|
||||
```
|
||||
config.json
|
||||
└── codeConfig[] # 代码生成配置数组(支持多套)
|
||||
├── config # 菜单权限配置文件路径(如 admin.json)
|
||||
├── configDB # 数据库生成的完整配置
|
||||
├── rule # 字段规则配置文件路径(如 rule.json)
|
||||
├── table # 管理员表名
|
||||
├── name # 生成代码的包名
|
||||
└── mode # 生成模式
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 一、config.json 中的 codeConfig
|
||||
|
||||
### 配置结构
|
||||
|
||||
```json
|
||||
{
|
||||
"codeConfig": [
|
||||
{
|
||||
"config": "config/admin.json",
|
||||
"configDB": "config/adminDB.json",
|
||||
"mode": 0,
|
||||
"name": "",
|
||||
"rule": "config/rule.json",
|
||||
"table": "admin"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 配置项说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| config | string | 菜单权限配置文件路径,用于定义菜单结构、权限控制 |
|
||||
| configDB | string | 代码生成器输出的完整配置文件(自动生成) |
|
||||
| rule | string | 字段规则配置文件路径,定义字段在增删改查中的行为 |
|
||||
| table | string | 管理员/用户表名,用于身份验证和权限控制 |
|
||||
| name | string | 生成的代码包名,为空则不生成代码文件 |
|
||||
| mode | int | 生成模式:0-仅配置不生成代码,非0-生成代码文件 |
|
||||
|
||||
### 多配置支持
|
||||
|
||||
可以配置多个独立的代码生成实例,适用于多端场景:
|
||||
|
||||
```json
|
||||
{
|
||||
"codeConfig": [
|
||||
{
|
||||
"config": "config/admin.json",
|
||||
"table": "admin",
|
||||
"rule": "config/rule.json"
|
||||
},
|
||||
{
|
||||
"config": "config/user.json",
|
||||
"table": "user",
|
||||
"rule": "config/rule.json"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、菜单权限配置(如 admin.json)
|
||||
|
||||
### 配置文件更新机制
|
||||
|
||||
- **首次运行**:代码生成器会根据数据库表结构自动创建配置文件(如 admin.json)
|
||||
- **后续运行**:配置文件**不会自动更新**,避免覆盖手动修改的内容
|
||||
- **重新生成**:如需重新生成,删除配置文件后重新运行即可
|
||||
- **参考更新**:可参考 `configDB` 指定的文件(如 adminDB.json)查看最新的数据库结构变化,手动调整配置
|
||||
|
||||
### 完整配置结构
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "唯一标识(自动生成)",
|
||||
"name": "admin",
|
||||
"label": "管理平台名称",
|
||||
"labelConfig": { ... },
|
||||
"menus": [ ... ],
|
||||
"flow": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### 2.1 label 配置
|
||||
|
||||
定义系统显示名称:
|
||||
|
||||
```json
|
||||
{
|
||||
"label": "HoTime管理平台"
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 labelConfig 配置
|
||||
|
||||
定义权限的显示文字:
|
||||
|
||||
```json
|
||||
{
|
||||
"labelConfig": {
|
||||
"show": "开启",
|
||||
"add": "添加",
|
||||
"delete": "删除",
|
||||
"edit": "编辑",
|
||||
"info": "查看详情",
|
||||
"download": "下载清单"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| 操作 | 说明 |
|
||||
|------|------|
|
||||
| show | 显示/查看列表权限 |
|
||||
| add | 添加数据权限 |
|
||||
| delete | 删除数据权限 |
|
||||
| edit | 编辑数据权限 |
|
||||
| info | 查看详情权限 |
|
||||
| download | 下载/导出权限 |
|
||||
|
||||
### 2.3 menus 配置
|
||||
|
||||
定义菜单结构,支持多级嵌套:
|
||||
|
||||
```json
|
||||
{
|
||||
"menus": [
|
||||
{
|
||||
"label": "系统管理",
|
||||
"name": "sys",
|
||||
"icon": "Setting",
|
||||
"auth": ["show"],
|
||||
"menus": [
|
||||
{
|
||||
"label": "用户管理",
|
||||
"table": "user",
|
||||
"auth": ["show", "add", "delete", "edit", "info", "download"]
|
||||
},
|
||||
{
|
||||
"label": "角色管理",
|
||||
"table": "role",
|
||||
"auth": ["show", "add", "delete", "edit", "info"]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"label": "文章管理",
|
||||
"table": "article",
|
||||
"icon": "Document",
|
||||
"auth": ["show", "add", "edit", "info"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### menus 字段说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| label | string | 菜单显示名称 |
|
||||
| name | string | 菜单标识(用于分组,不绑定表时使用) |
|
||||
| table | string | 绑定的数据表名(用于自动生成 CRUD) |
|
||||
| icon | string | 菜单图标名称 |
|
||||
| auth | array | 权限数组,定义该菜单/表拥有的操作权限 |
|
||||
| menus | array | 子菜单数组(支持嵌套) |
|
||||
|
||||
#### name vs table
|
||||
|
||||
- **table**: 绑定数据表,拥有该表的增删查改等权限,自动生成 CRUD 接口
|
||||
- **name**: 自定义功能标识,不绑定表,前端根据 name 和 auth 来显示和操作自定义内容(如首页 home、仪表盘 dashboard 等)
|
||||
- 如果配置了 `menus` 子菜单,则作为分组功能,前端展开显示下级菜单
|
||||
- 如果没有 `menus`,则作为独立的自定义功能入口
|
||||
|
||||
**注意**:目前只支持**两级菜单**,不允许更多层级嵌套。
|
||||
|
||||
```json
|
||||
// 使用 table:绑定数据表,有增删查改权限
|
||||
{ "label": "用户管理", "table": "user", "auth": ["show", "add", "edit", "delete"] }
|
||||
|
||||
// 使用 name:自定义功能(无子菜单)
|
||||
{ "label": "首页", "name": "home", "icon": "House", "auth": ["show"] }
|
||||
|
||||
// 使用 name:分组功能(有子菜单)
|
||||
{ "label": "系统管理", "name": "sys", "icon": "Setting", "auth": ["show"], "menus": [...] }
|
||||
```
|
||||
|
||||
#### 自动分组规则
|
||||
|
||||
代码生成器在自动生成配置时,会根据表名的 `_` 分词进行自动分组:
|
||||
|
||||
- 表名 `sys_logs`、`sys_menus`、`sys_config` → 自动归入 `sys` 分组
|
||||
- 表名 `article`、`article_tag` → 自动归入 `article` 分组
|
||||
|
||||
分组的显示名称(label)使用该分组下**第一张表的名字**,如果表有备注则使用备注名。
|
||||
|
||||
### 2.4 auth 配置
|
||||
|
||||
权限数组定义菜单/表拥有的操作权限:
|
||||
|
||||
```json
|
||||
{
|
||||
"auth": ["show", "add", "delete", "edit", "info", "download"]
|
||||
}
|
||||
```
|
||||
|
||||
#### 内置权限
|
||||
|
||||
| 权限 | 对应接口 | 说明 |
|
||||
|------|----------|------|
|
||||
| show | /search | 列表查询 |
|
||||
| add | /add | 新增数据 |
|
||||
| delete | /remove | 删除数据 |
|
||||
| edit | /update | 编辑数据 |
|
||||
| info | /info | 查看详情 |
|
||||
| download | /search?download=1 | 导出数据 |
|
||||
|
||||
#### 自定义权限扩展
|
||||
|
||||
auth 数组**可以自由增删**,新增的权限项会:
|
||||
- 在前端菜单/功能中显示
|
||||
- 在角色管理(role)的权限设置中显示,供管理员分配
|
||||
|
||||
```json
|
||||
// 示例:为文章表添加自定义权限
|
||||
{
|
||||
"label": "文章管理",
|
||||
"table": "article",
|
||||
"auth": ["show", "add", "edit", "delete", "info", "publish", "audit", "top"]
|
||||
}
|
||||
```
|
||||
|
||||
上例中 `publish`(发布)、`audit`(审核)、`top`(置顶)为自定义权限,前端可根据这些权限控制对应按钮的显示和操作。
|
||||
|
||||
### 2.5 icon 配置
|
||||
|
||||
菜单图标,使用 Element Plus 图标名称:
|
||||
|
||||
```json
|
||||
{ "icon": "Setting" } // 设置图标
|
||||
{ "icon": "User" } // 用户图标
|
||||
{ "icon": "Document" } // 文档图标
|
||||
{ "icon": "Folder" } // 文件夹图标
|
||||
```
|
||||
|
||||
### 2.6 flow 配置
|
||||
|
||||
flow 是一个简易的数据权限控制机制,用于限制用户只能操作自己权限范围内的数据。
|
||||
|
||||
#### 基本结构
|
||||
|
||||
```json
|
||||
{
|
||||
"flow": {
|
||||
"role": {
|
||||
"table": "role",
|
||||
"stop": true,
|
||||
"sql": {
|
||||
"id": "role_id"
|
||||
}
|
||||
},
|
||||
"article": {
|
||||
"table": "article",
|
||||
"stop": false,
|
||||
"sql": {
|
||||
"admin_id": "id"
|
||||
}
|
||||
},
|
||||
"org": {
|
||||
"table": "org",
|
||||
"stop": false,
|
||||
"sql": {
|
||||
"parent_ids[~]": ",org_id,"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### flow 字段说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| table | string | 表名 |
|
||||
| stop | bool | 是否禁止修改自身关联的数据 |
|
||||
| sql | object | 数据过滤条件,自动填充查询/操作条件 |
|
||||
|
||||
#### stop 配置详解
|
||||
|
||||
`stop` 用于防止用户修改自己当前关联的敏感数据。
|
||||
|
||||
**场景示例**:当前登录用户是 admin 表的用户,其 `role_id = 1`
|
||||
|
||||
```json
|
||||
"role": {
|
||||
"table": "role",
|
||||
"stop": true,
|
||||
"sql": { "id": "role_id" }
|
||||
}
|
||||
```
|
||||
|
||||
**效果**:
|
||||
- 用户**不能修改** role 表中 `id = 1` 的这行数据(自己的角色)
|
||||
- 用户**可以修改**其他 role 记录(如果有权限的话)
|
||||
|
||||
**典型用途**:
|
||||
- 防止用户提升自己的角色权限
|
||||
- 防止用户修改自己所属的组织
|
||||
|
||||
#### sql 配置详解
|
||||
|
||||
`sql` 用于自动填充数据过滤条件,实现数据隔离。
|
||||
|
||||
**格式**:`{ "目标表字段": "当前用户字段" }`
|
||||
|
||||
**示例 1:精确匹配**
|
||||
|
||||
```json
|
||||
"article": {
|
||||
"sql": { "admin_id": "id" }
|
||||
}
|
||||
```
|
||||
|
||||
**效果**:查询/操作 article 表时,自动添加条件 `WHERE admin_id = 当前用户.id`
|
||||
|
||||
即:用户只能看到/操作自己创建的文章。
|
||||
|
||||
**示例 2:角色关联**
|
||||
|
||||
```json
|
||||
"role": {
|
||||
"sql": { "id": "role_id" }
|
||||
}
|
||||
```
|
||||
|
||||
**效果**:查询/操作 role 表时,自动添加条件 `WHERE id = 当前用户.role_id`
|
||||
|
||||
即:用户只能看到自己的角色。
|
||||
|
||||
**示例 3:树形结构(模糊匹配)**
|
||||
|
||||
```json
|
||||
"org": {
|
||||
"sql": { "parent_ids[~]": ",org_id," }
|
||||
}
|
||||
```
|
||||
|
||||
**效果**:查询/操作 org 表时,自动添加条件 `WHERE parent_ids LIKE '%,用户.org_id,%'`
|
||||
|
||||
即:用户只能看到自己组织及其下级组织。
|
||||
|
||||
#### sql 条件语法
|
||||
|
||||
| 格式 | 含义 | SQL 等价 |
|
||||
|------|------|----------|
|
||||
| `"field": "user_field"` | 精确匹配 | `field = 用户.user_field` |
|
||||
| `"field[~]": ",value,"` | 模糊匹配 | `field LIKE '%,value,%'` |
|
||||
|
||||
#### 完整示例
|
||||
|
||||
假设当前登录用户数据:
|
||||
```json
|
||||
{ "id": 5, "role_id": 2, "org_id": 10 }
|
||||
```
|
||||
|
||||
flow 配置:
|
||||
```json
|
||||
{
|
||||
"flow": {
|
||||
"role": {
|
||||
"table": "role",
|
||||
"stop": true,
|
||||
"sql": { "id": "role_id" }
|
||||
},
|
||||
"article": {
|
||||
"table": "article",
|
||||
"stop": false,
|
||||
"sql": { "admin_id": "id" }
|
||||
},
|
||||
"org": {
|
||||
"table": "org",
|
||||
"stop": false,
|
||||
"sql": { "parent_ids[~]": ",org_id," }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**效果**:
|
||||
|
||||
| 表 | 查询条件 | stop 效果 |
|
||||
|-----|----------|-----------|
|
||||
| role | `WHERE id = 2` | 不能修改 id=2 的角色 |
|
||||
| article | `WHERE admin_id = 5` | 可以修改自己的文章 |
|
||||
| org | `WHERE parent_ids LIKE '%,10,%'` | 可以修改下级组织 |
|
||||
|
||||
---
|
||||
|
||||
## 三、字段规则配置(rule.json)
|
||||
|
||||
定义字段在增删改查操作中的默认行为。
|
||||
|
||||
### 配置结构
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"name": "id",
|
||||
"add": false,
|
||||
"edit": false,
|
||||
"info": true,
|
||||
"list": true,
|
||||
"must": false,
|
||||
"strict": true,
|
||||
"type": ""
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### 字段属性说明
|
||||
|
||||
| 属性 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| name | string | 字段名或字段名包含的关键词 |
|
||||
| add | bool | 新增时是否显示该字段 |
|
||||
| edit | bool | 编辑时是否显示该字段 |
|
||||
| info | bool | 详情页是否显示该字段 |
|
||||
| list | bool | 列表页是否显示该字段 |
|
||||
| must | bool | 是否必填(详见下方说明) |
|
||||
| strict | bool | 是否严格匹配字段名(true=完全匹配,false=包含匹配) |
|
||||
| type | string | 字段类型(影响前端控件和数据处理) |
|
||||
|
||||
### must 必填字段规则
|
||||
|
||||
`must` 字段用于控制前端表单的必填验证:
|
||||
|
||||
**自动识别规则**:
|
||||
1. **MySQL**:如果字段设置为 `NOT NULL`(即 `IS_NULLABLE='NO'`),自动设为 `must=true`
|
||||
2. **SQLite**:如果字段是主键(`pk=1`),自动设为 `must=true`
|
||||
|
||||
**规则配置覆盖**:
|
||||
- `rule.json` 中的 `must` 设置会覆盖数据库的自动识别结果
|
||||
- 可以将数据库中 NOT NULL 的字段在规则中设为 `must=false`,反之亦然
|
||||
|
||||
**前端效果**:
|
||||
- `must=true` 的字段在新增/编辑表单中显示必填标记(*)
|
||||
- 提交时前端会验证必填字段
|
||||
|
||||
**后端验证**:
|
||||
- 新增操作时,如果 `must=true` 的字段为空,返回"请求参数不足"
|
||||
|
||||
### type 类型说明
|
||||
|
||||
| 类型 | 说明 | 前端控件 |
|
||||
|------|------|----------|
|
||||
| (空) | 普通文本 | 文本输入框 |
|
||||
| text | 文本 | 文本输入框 |
|
||||
| number | 数字 | 数字输入框 |
|
||||
| select | 选择 | 下拉选择框(根据注释自动生成选项) |
|
||||
| time | 时间(datetime) | 日期时间选择器 |
|
||||
| unixTime | 时间戳 | 日期时间选择器(存储为 Unix 时间戳) |
|
||||
| password | 密码 | 密码输入框(自动 MD5 加密) |
|
||||
| textArea | 多行文本 | 文本域 |
|
||||
| image | 图片 | 图片上传 |
|
||||
| file | 文件 | 文件上传 |
|
||||
| money | 金额 | 金额输入框 |
|
||||
| auth | 权限 | 权限树选择器 |
|
||||
| form | 表单 | 动态表单 |
|
||||
| index | 索引 | 隐藏字段(用于 parent_ids 等) |
|
||||
| table | 动态表 | 表名选择器 |
|
||||
| table_id | 动态表ID | 根据 table 字段动态关联 |
|
||||
|
||||
### 内置字段规则
|
||||
|
||||
以下是框架默认的字段规则,可在 `rule.json` 中覆盖:
|
||||
|
||||
#### 主键和索引
|
||||
|
||||
```json
|
||||
{"name": "id", "add": false, "list": true, "edit": false, "info": true, "strict": true}
|
||||
{"name": "sn", "add": false, "list": true, "edit": false, "info": true}
|
||||
{"name": "parent_ids", "add": false, "list": false, "edit": false, "info": false, "type": "index", "strict": true}
|
||||
{"name": "index", "add": false, "list": false, "edit": false, "info": false, "type": "index", "strict": true}
|
||||
```
|
||||
|
||||
#### 层级关系
|
||||
|
||||
```json
|
||||
{"name": "parent_id", "add": true, "list": true, "edit": true, "info": true}
|
||||
{"name": "level", "add": false, "list": false, "edit": false, "info": true}
|
||||
```
|
||||
|
||||
#### 时间字段
|
||||
|
||||
```json
|
||||
{"name": "create_time", "add": false, "list": false, "edit": false, "info": true, "type": "time", "strict": true}
|
||||
{"name": "modify_time", "add": false, "list": true, "edit": false, "info": true, "type": "time", "strict": true}
|
||||
{"name": "time", "add": true, "list": true, "edit": true, "info": true, "type": "time"}
|
||||
```
|
||||
|
||||
#### 状态字段
|
||||
|
||||
```json
|
||||
{"name": "status", "add": true, "list": true, "edit": true, "info": true, "type": "select"}
|
||||
{"name": "state", "add": true, "list": true, "edit": true, "info": true, "type": "select"}
|
||||
{"name": "sex", "add": true, "list": true, "edit": true, "info": true, "type": "select"}
|
||||
```
|
||||
|
||||
#### 敏感字段
|
||||
|
||||
```json
|
||||
{"name": "password", "add": true, "list": false, "edit": true, "info": false, "type": "password"}
|
||||
{"name": "pwd", "add": true, "list": false, "edit": true, "info": false, "type": "password"}
|
||||
{"name": "delete", "add": false, "list": false, "edit": false, "info": false}
|
||||
{"name": "version", "add": false, "list": false, "edit": false, "info": false}
|
||||
```
|
||||
|
||||
#### 媒体字段
|
||||
|
||||
```json
|
||||
{"name": "image", "add": true, "list": false, "edit": true, "info": true, "type": "image"}
|
||||
{"name": "img", "add": true, "list": false, "edit": true, "info": true, "type": "image"}
|
||||
{"name": "avatar", "add": true, "list": false, "edit": true, "info": true, "type": "image"}
|
||||
{"name": "icon", "add": true, "list": false, "edit": true, "info": true, "type": "image"}
|
||||
{"name": "file", "add": true, "list": false, "edit": true, "info": true, "type": "file"}
|
||||
```
|
||||
|
||||
#### 文本字段
|
||||
|
||||
```json
|
||||
{"name": "info", "add": true, "list": false, "edit": true, "info": true, "type": "textArea"}
|
||||
{"name": "content", "add": true, "list": false, "edit": true, "info": true, "type": "textArea"}
|
||||
{"name": "description", "add": true, "list": false, "edit": true, "info": true}
|
||||
{"name": "note", "add": true, "list": false, "edit": true, "info": true}
|
||||
{"name": "address", "add": true, "list": true, "edit": true, "info": true}
|
||||
```
|
||||
|
||||
#### 特殊字段
|
||||
|
||||
```json
|
||||
{"name": "amount", "add": true, "list": true, "edit": true, "info": true, "type": "money", "strict": true}
|
||||
{"name": "auth", "add": true, "list": false, "edit": true, "info": true, "type": "auth", "strict": true}
|
||||
{"name": "rule", "add": true, "list": true, "edit": true, "info": true, "type": "form"}
|
||||
{"name": "table", "add": false, "list": true, "edit": false, "info": true, "type": "table"}
|
||||
{"name": "table_id", "add": false, "list": true, "edit": false, "info": true, "type": "table_id"}
|
||||
```
|
||||
|
||||
### 自定义字段规则
|
||||
|
||||
在 `rule.json` 中添加项目特定的字段规则:
|
||||
|
||||
```json
|
||||
[
|
||||
// 项目特定规则
|
||||
{
|
||||
"name": "company_name",
|
||||
"add": true,
|
||||
"edit": true,
|
||||
"info": true,
|
||||
"list": true,
|
||||
"must": true,
|
||||
"strict": true,
|
||||
"type": ""
|
||||
},
|
||||
// 表.字段 形式的精确规则
|
||||
{
|
||||
"name": "user.nickname",
|
||||
"add": true,
|
||||
"edit": true,
|
||||
"info": true,
|
||||
"list": true,
|
||||
"strict": true,
|
||||
"type": ""
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、配置示例
|
||||
|
||||
### 完整的 admin.json 示例
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "74a8a59407fa7d6c7fcdc85742dbae57",
|
||||
"name": "admin",
|
||||
"label": "后台管理系统",
|
||||
"labelConfig": {
|
||||
"show": "开启",
|
||||
"add": "添加",
|
||||
"delete": "删除",
|
||||
"edit": "编辑",
|
||||
"info": "查看详情",
|
||||
"download": "下载清单"
|
||||
},
|
||||
"menus": [
|
||||
{
|
||||
"label": "系统管理",
|
||||
"name": "sys",
|
||||
"icon": "Setting",
|
||||
"auth": ["show"],
|
||||
"menus": [
|
||||
{
|
||||
"label": "日志管理",
|
||||
"table": "logs",
|
||||
"auth": ["show", "download"]
|
||||
},
|
||||
{
|
||||
"label": "角色管理",
|
||||
"table": "role",
|
||||
"auth": ["show", "add", "delete", "edit", "info"]
|
||||
},
|
||||
{
|
||||
"label": "组织管理",
|
||||
"table": "org",
|
||||
"auth": ["show", "add", "delete", "edit", "info"]
|
||||
},
|
||||
{
|
||||
"label": "人员管理",
|
||||
"table": "admin",
|
||||
"auth": ["show", "add", "delete", "edit", "info", "download"]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"flow": {
|
||||
"admin": {
|
||||
"table": "admin",
|
||||
"stop": false,
|
||||
"sql": { "role_id": "role_id" }
|
||||
},
|
||||
"role": {
|
||||
"table": "role",
|
||||
"stop": true,
|
||||
"sql": { "admin_id": "id", "id": "role_id" }
|
||||
},
|
||||
"org": {
|
||||
"table": "org",
|
||||
"stop": false,
|
||||
"sql": { "admin_id": "id" }
|
||||
},
|
||||
"logs": {
|
||||
"table": "logs",
|
||||
"stop": false,
|
||||
"sql": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、SQLite 备注替代方案
|
||||
|
||||
SQLite 数据库不支持表备注(TABLE COMMENT)和字段备注(COLUMN COMMENT),代码生成器会使用表名/字段名作为默认显示名称。
|
||||
|
||||
### 通过配置文件设置备注
|
||||
|
||||
利用 HoTime 的配置覆盖机制,可以在配置文件中手动设置显示名称和提示:
|
||||
|
||||
**步骤**:
|
||||
1. 首次运行,生成配置文件(如 admin.json)
|
||||
2. 编辑配置文件中的 `tables` 部分
|
||||
|
||||
### 设置表显示名称
|
||||
|
||||
在菜单配置中设置 `label`:
|
||||
|
||||
```json
|
||||
{
|
||||
"menus": [
|
||||
{
|
||||
"label": "用户管理", // 手动设置表显示名称
|
||||
"table": "user",
|
||||
"auth": ["show", "add", "edit", "delete"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 设置字段显示名称和提示
|
||||
|
||||
在 `configDB` 文件(如 adminDB.json)中的 `tables.表名.columns` 部分设置:
|
||||
|
||||
```json
|
||||
{
|
||||
"tables": {
|
||||
"user": {
|
||||
"label": "用户管理",
|
||||
"columns": [
|
||||
{
|
||||
"name": "id",
|
||||
"label": "ID",
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"label": "用户名",
|
||||
"ps": "请输入用户名", // 前端输入提示
|
||||
"must": true
|
||||
},
|
||||
{
|
||||
"name": "phone",
|
||||
"label": "手机号",
|
||||
"ps": "请输入11位手机号"
|
||||
},
|
||||
{
|
||||
"name": "status",
|
||||
"label": "状态",
|
||||
"type": "select",
|
||||
"options": [
|
||||
{"name": "正常", "value": "0"},
|
||||
{"name": "禁用", "value": "1"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**注意**:直接编辑的是 `configDB` 指定的文件(如 adminDB.json),该文件会在每次启动时重新生成。如需持久化修改,应将自定义的 columns 配置放入 `config` 指定的文件(如 admin.json)中。
|
||||
|
||||
---
|
||||
|
||||
## 六、配置检查清单
|
||||
|
||||
### codeConfig 检查
|
||||
|
||||
- [ ] config 文件路径正确
|
||||
- [ ] rule 文件路径正确
|
||||
- [ ] table 指定的管理员表存在
|
||||
|
||||
### 菜单权限配置检查
|
||||
|
||||
- [ ] 所有 table 指向的表在数据库中存在
|
||||
- [ ] auth 数组包含需要的权限
|
||||
- [ ] menus 结构正确(有子菜单用 name,无子菜单用 table)
|
||||
- [ ] flow 配置的 sql 条件字段存在
|
||||
|
||||
### 字段规则配置检查
|
||||
|
||||
- [ ] strict=true 的规则字段名完全匹配
|
||||
- [ ] type 类型与前端控件需求一致
|
||||
- [ ] 敏感字段(password等)的 list 和 info 为 false
|
||||
@@ -0,0 +1,408 @@
|
||||
# 数据库设计规范
|
||||
|
||||
本文档定义了 HoTime 框架代码生成器所依赖的数据库设计规范。遵循这些规范可以确保代码生成器正确识别表关系、自动生成 CRUD 接口和管理后台。
|
||||
|
||||
---
|
||||
|
||||
## 表命名规则
|
||||
|
||||
### 1. 关于表名前缀
|
||||
|
||||
一般情况下**没必要强行添加前缀分组**,直接使用业务含义命名即可:
|
||||
|
||||
```
|
||||
推荐:user、org、role、article
|
||||
```
|
||||
|
||||
**可以使用前缀的场景**:当项目较大、表较多时,可以用前缀进行模块分组(如 `sys_`、`cms_`),代码生成器会根据 `_` 分词自动将同前缀的表归为一组。
|
||||
|
||||
```
|
||||
sys_user、sys_role、sys_org → 自动归入 sys 分组
|
||||
cms_article、cms_category → 自动归入 cms 分组
|
||||
```
|
||||
|
||||
**使用前缀的注意事项**:
|
||||
- 必须遵循外键全局唯一性规则(见下文)
|
||||
- 前缀分组后,关联 ID 命名会更复杂,容易产生重复
|
||||
- **外键名不允许重复指向不同的表**
|
||||
|
||||
### 2. 可使用简称
|
||||
|
||||
较长的表名可以使用常见简称:
|
||||
|
||||
| 全称 | 简称 |
|
||||
|------|------|
|
||||
| organization | org |
|
||||
| category | ctg |
|
||||
| configuration | config |
|
||||
| administrator | admin |
|
||||
|
||||
### 3. 关联表命名
|
||||
|
||||
多对多关联表使用 `主表_关联表` 格式:
|
||||
|
||||
```
|
||||
user_org -- 用户与组织的关联
|
||||
user_role -- 用户与角色的关联
|
||||
article_tag -- 文章与标签的关联
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 字段命名规则
|
||||
|
||||
### 1. 主键字段
|
||||
|
||||
所有表的主键统一命名为 `id`,使用自增整数。
|
||||
|
||||
```sql
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT
|
||||
```
|
||||
|
||||
### 2. 外键字段
|
||||
|
||||
外键使用 `完整表名_id` 格式:
|
||||
|
||||
```
|
||||
user_id -- 指向 user 表
|
||||
org_id -- 指向 org 表
|
||||
role_id -- 指向 role 表
|
||||
app_category_id -- 指向 app_category 表
|
||||
```
|
||||
|
||||
### 3. 关联表引用
|
||||
|
||||
引用关联表时使用 `关联表名_id`:
|
||||
|
||||
```
|
||||
user_org_id -- 指向 user_org 表
|
||||
user_role_id -- 指向 user_role 表
|
||||
```
|
||||
|
||||
### 4. 外键全局唯一性(重要)
|
||||
|
||||
**每个 `xxx_id` 字段名必须全局唯一指向一张表**,代码生成器通过 `_id` 后缀自动识别外键关系。
|
||||
|
||||
```
|
||||
✅ 正确设计:
|
||||
- user_id 只能指向 user 表
|
||||
- org_id 只能指向 org 表
|
||||
- user_org_id 只能指向 user_org 表
|
||||
- sys_user_id 只能指向 sys_user 表(如果使用前缀分组)
|
||||
|
||||
❌ 错误设计:
|
||||
- dd_id 作为"钉钉外部系统ID",但系统中存在 dd 表
|
||||
→ 代码生成器会误判为指向 dd 表的外键
|
||||
- 同时存在 user 表和 sys_user 表,都使用 user_id
|
||||
→ 外键名重复,代码生成器无法正确识别
|
||||
```
|
||||
|
||||
**使用前缀分组时的外键命名**:
|
||||
|
||||
如果使用了表名前缀(如 `sys_user`),外键应使用完整表名:
|
||||
|
||||
| 表名 | 外键命名 |
|
||||
|------|----------|
|
||||
| sys_user | sys_user_id |
|
||||
| sys_org | sys_org_id |
|
||||
| cms_article | cms_article_id |
|
||||
|
||||
**非外键业务标识字段**:避免使用 `xxx_id` 格式
|
||||
|
||||
| 业务含义 | 错误命名 | 正确命名 |
|
||||
|----------|----------|----------|
|
||||
| 设备唯一标识 | device_id | device_uuid / device_sn |
|
||||
| 钉钉用户ID | dingtalk_user_id | dt_user_id / dingtalk_uid |
|
||||
| 微信OpenID | wechat_id | wechat_openid |
|
||||
| 外部系统编号 | external_id | external_code / external_sn |
|
||||
|
||||
### 5. 外键智能匹配机制
|
||||
|
||||
代码生成器会按以下优先级自动匹配外键关联的表:
|
||||
|
||||
1. **完全匹配**:`user_id` → 查找 `user` 表
|
||||
2. **带前缀的完全匹配**:如果没找到,尝试 `user_id` → 查找 `sys_user` 表(默认前缀)
|
||||
3. **去除字段前缀匹配**:如果字段有前缀且找不到对应表,会去掉前缀匹配
|
||||
|
||||
**示例**:`admin_user_id` 字段的匹配过程:
|
||||
1. 先查找 `admin_user` 表 → 如果存在,关联到 `admin_user` 表
|
||||
2. 如果不存在,去掉前缀查找 `user` 表 → 如果存在,关联到 `user` 表
|
||||
|
||||
**使用场景**:当一张表需要记录多个用户(如创建人、审核人、处理人)时:
|
||||
|
||||
```sql
|
||||
CREATE TABLE `order` (
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT,
|
||||
`user_id` int(11) COMMENT '下单用户',
|
||||
`audit_user_id` int(11) COMMENT '审核人',
|
||||
`handle_user_id` int(11) COMMENT '处理人',
|
||||
...
|
||||
);
|
||||
```
|
||||
|
||||
如果系统中没有 `audit_user` 和 `handle_user` 表,代码生成器会自动将 `audit_user_id` 和 `handle_user_id` 识别为指向 `user` 表的外键。
|
||||
|
||||
### 6. 关联表冗余外键
|
||||
|
||||
关联表应包含必要的冗余外键便于查询:
|
||||
|
||||
```sql
|
||||
-- user_app 表(用户与应用的关联)
|
||||
CREATE TABLE user_app (
|
||||
id int(11) NOT NULL AUTO_INCREMENT,
|
||||
user_id int(11) DEFAULT NULL COMMENT '用户ID',
|
||||
app_id int(11) DEFAULT NULL COMMENT '应用ID',
|
||||
app_category_id int(11) DEFAULT NULL COMMENT '应用分类ID(冗余,便于按分类查询)',
|
||||
...
|
||||
);
|
||||
```
|
||||
|
||||
### 7. 层级关系字段
|
||||
|
||||
树形结构的表使用以下字段:
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| parent_id | int | 父级ID,顶级为 NULL 或 0 |
|
||||
| parent_ids | varchar(255) | 完整层级路径,格式:`,1,3,5,`(从根到当前,逗号分隔) |
|
||||
| level | int | 层级深度,从 0 开始 |
|
||||
|
||||
**示例**:
|
||||
|
||||
```
|
||||
id=1, parent_id=NULL, parent_ids=",1,", level=0 -- 顶级
|
||||
id=3, parent_id=1, parent_ids=",1,3,", level=1 -- 一级
|
||||
id=5, parent_id=3, parent_ids=",1,3,5,", level=2 -- 二级
|
||||
```
|
||||
|
||||
`parent_ids` 的设计便于快速查询所有子级:
|
||||
```sql
|
||||
SELECT * FROM org WHERE parent_ids LIKE '%,3,%' -- 查询id=3的所有子级
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 时间字段类型
|
||||
|
||||
不同数据库使用对应的时间类型:
|
||||
|
||||
| 数据库 | 类型 | 示例 |
|
||||
|--------|------|------|
|
||||
| MySQL | datetime | `2024-01-15 10:30:00` |
|
||||
| SQLite | TEXT | `2024-01-15 10:30:00` |
|
||||
| PostgreSQL | timestamp | `2024-01-15 10:30:00` |
|
||||
|
||||
---
|
||||
|
||||
## 表注释规则
|
||||
|
||||
### 表备注命名建议
|
||||
|
||||
表备注建议使用"XX管理"格式,便于在管理后台显示:
|
||||
|
||||
```sql
|
||||
-- MySQL 表备注示例
|
||||
CREATE TABLE `user` (
|
||||
...
|
||||
) COMMENT='用户管理';
|
||||
|
||||
CREATE TABLE `article` (
|
||||
...
|
||||
) COMMENT='文章管理';
|
||||
|
||||
CREATE TABLE `order` (
|
||||
...
|
||||
) COMMENT='订单管理';
|
||||
```
|
||||
|
||||
### SQLite 表备注
|
||||
|
||||
SQLite 不支持表备注,代码生成器会使用表名作为默认显示名称。如需自定义,可在配置文件中手动设置 `label`(详见代码生成配置规范)。
|
||||
|
||||
---
|
||||
|
||||
## 字段注释规则
|
||||
|
||||
### 注释语法格式
|
||||
|
||||
字段注释支持以下格式组合:
|
||||
|
||||
```
|
||||
显示名称:选项值 附加备注{前端提示}
|
||||
```
|
||||
|
||||
| 部分 | 分隔符 | 用途 |
|
||||
|------|--------|------|
|
||||
| 显示名称 | 无 | 前端表单/列表的字段标签 |
|
||||
| 选项值 | `:` 冒号 | select 类型的下拉选项(格式:`值-名称,值-名称`) |
|
||||
| 附加备注 | 空格 | 仅在数据库中查看,不传递给前端 |
|
||||
| 前端提示 | `{}` | 存储到 `ps` 字段,用于前端显示提示文字 |
|
||||
|
||||
### 选择类型字段
|
||||
|
||||
使用 `标签:值-名称,值-名称` 格式,代码生成器会自动解析为下拉选项:
|
||||
|
||||
```sql
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏'
|
||||
`sex` int(1) DEFAULT '0' COMMENT '性别:0-未知,1-男,2-女'
|
||||
`status` int(2) DEFAULT '0' COMMENT '审核状态:0-待审核,1-已通过,2-已拒绝'
|
||||
```
|
||||
|
||||
### 普通字段
|
||||
|
||||
直接使用中文说明:
|
||||
|
||||
```sql
|
||||
`name` varchar(50) DEFAULT NULL COMMENT '名称'
|
||||
`phone` varchar(20) DEFAULT NULL COMMENT '手机号'
|
||||
`email` varchar(100) DEFAULT NULL COMMENT '邮箱'
|
||||
```
|
||||
|
||||
### 附加备注(空格后)
|
||||
|
||||
空格后的内容**不会传递给前端**,仅供数据库设计时参考:
|
||||
|
||||
```sql
|
||||
`user_id` int(11) COMMENT '用户ID 关联user表的主键'
|
||||
`amount` decimal(10,2) COMMENT '金额 单位为元,精确到分'
|
||||
```
|
||||
|
||||
### 前端提示({}内)
|
||||
|
||||
`{}` 中的内容存储到 `ps` 字段,前端可用于显示输入提示:
|
||||
|
||||
```sql
|
||||
`phone` varchar(20) COMMENT '手机号{请输入11位手机号}'
|
||||
`email` varchar(100) COMMENT '邮箱{格式:xxx@xxx.com}'
|
||||
`content` text COMMENT '内容{支持HTML格式}'
|
||||
```
|
||||
|
||||
### 组合使用
|
||||
|
||||
```sql
|
||||
-- 完整格式示例
|
||||
`status` int(2) DEFAULT '0' COMMENT '状态:0-待审核,1-已通过,2-已拒绝 业务状态流转字段{选择当前审核状态}'
|
||||
```
|
||||
|
||||
解析结果:
|
||||
- `label` = "状态"
|
||||
- `options` = [{name:"待审核", value:"0"}, {name:"已通过", value:"1"}, {name:"已拒绝", value:"2"}]
|
||||
- `ps` = "选择当前审核状态"
|
||||
- 空格后的"业务状态流转字段"不会传递给前端
|
||||
|
||||
### SQLite 字段备注
|
||||
|
||||
SQLite 不支持字段备注(COMMENT),代码生成器会使用字段名作为默认显示名称。如需自定义,可在配置文件中手动设置(详见代码生成配置规范)。
|
||||
|
||||
---
|
||||
|
||||
## 必有字段规则
|
||||
|
||||
每张业务表必须包含以下三个字段:
|
||||
|
||||
### MySQL
|
||||
|
||||
```sql
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏',
|
||||
`create_time` datetime DEFAULT NULL COMMENT '创建日期',
|
||||
`modify_time` datetime DEFAULT NULL COMMENT '变更时间',
|
||||
```
|
||||
|
||||
### SQLite
|
||||
|
||||
```sql
|
||||
"state" INTEGER DEFAULT 0, -- 状态:0-正常,1-异常,2-隐藏
|
||||
"create_time" TEXT DEFAULT NULL, -- 创建日期
|
||||
"modify_time" TEXT DEFAULT NULL, -- 变更时间
|
||||
```
|
||||
|
||||
### PostgreSQL
|
||||
|
||||
```sql
|
||||
"state" INTEGER DEFAULT 0, -- 状态:0-正常,1-异常,2-隐藏
|
||||
"create_time" TIMESTAMP DEFAULT NULL, -- 创建日期
|
||||
"modify_time" TIMESTAMP DEFAULT NULL, -- 变更时间
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 完整建表示例
|
||||
|
||||
### MySQL 示例
|
||||
|
||||
```sql
|
||||
-- 用户表
|
||||
CREATE TABLE `user` (
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '主键ID',
|
||||
`name` varchar(50) DEFAULT NULL COMMENT '用户名',
|
||||
`phone` varchar(20) DEFAULT NULL COMMENT '手机号',
|
||||
`password` varchar(64) DEFAULT NULL COMMENT '密码',
|
||||
`org_id` int(11) DEFAULT NULL COMMENT '组织ID',
|
||||
`role_id` int(11) DEFAULT NULL COMMENT '角色ID',
|
||||
`avatar` varchar(255) DEFAULT NULL COMMENT '头像',
|
||||
`sex` int(1) DEFAULT '0' COMMENT '性别:0-未知,1-男,2-女',
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏',
|
||||
`create_time` datetime DEFAULT NULL COMMENT '创建日期',
|
||||
`modify_time` datetime DEFAULT NULL COMMENT '变更时间',
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='用户表';
|
||||
|
||||
-- 组织表(树形结构)
|
||||
CREATE TABLE `org` (
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '主键ID',
|
||||
`name` varchar(100) DEFAULT NULL COMMENT '组织名称',
|
||||
`parent_id` int(11) DEFAULT NULL COMMENT '父级ID',
|
||||
`parent_ids` varchar(255) DEFAULT NULL COMMENT '层级路径',
|
||||
`level` int(2) DEFAULT '0' COMMENT '层级深度',
|
||||
`sort` int(5) DEFAULT '0' COMMENT '排序',
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏',
|
||||
`create_time` datetime DEFAULT NULL COMMENT '创建日期',
|
||||
`modify_time` datetime DEFAULT NULL COMMENT '变更时间',
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='组织表';
|
||||
|
||||
-- 用户组织关联表
|
||||
CREATE TABLE `user_org` (
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '主键ID',
|
||||
`user_id` int(11) DEFAULT NULL COMMENT '用户ID',
|
||||
`org_id` int(11) DEFAULT NULL COMMENT '组织ID',
|
||||
`state` int(2) DEFAULT '0' COMMENT '状态:0-正常,1-异常,2-隐藏',
|
||||
`create_time` datetime DEFAULT NULL COMMENT '创建日期',
|
||||
`modify_time` datetime DEFAULT NULL COMMENT '变更时间',
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='用户组织关联表';
|
||||
```
|
||||
|
||||
### SQLite 示例
|
||||
|
||||
```sql
|
||||
-- 用户表
|
||||
CREATE TABLE "user" (
|
||||
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
"name" TEXT DEFAULT NULL,
|
||||
"phone" TEXT DEFAULT NULL,
|
||||
"password" TEXT DEFAULT NULL,
|
||||
"org_id" INTEGER DEFAULT NULL,
|
||||
"role_id" INTEGER DEFAULT NULL,
|
||||
"avatar" TEXT DEFAULT NULL,
|
||||
"sex" INTEGER DEFAULT 0,
|
||||
"state" INTEGER DEFAULT 0,
|
||||
"create_time" TEXT DEFAULT NULL,
|
||||
"modify_time" TEXT DEFAULT NULL
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 规范检查清单
|
||||
|
||||
在设计数据库时,请确认:
|
||||
|
||||
- [ ] 表名无系统前缀
|
||||
- [ ] 主键统一为 `id`
|
||||
- [ ] 外键格式为 `表名_id`
|
||||
- [ ] 所有 `xxx_id` 字段都指向实际存在的表
|
||||
- [ ] 非外键业务标识不使用 `xxx_id` 格式
|
||||
- [ ] 树形表包含 parent_id、parent_ids、level
|
||||
- [ ] 所有表包含 state、create_time、modify_time
|
||||
- [ ] 选择字段注释格式正确(`标签:值-名称`)
|
||||
+80
-5
@@ -135,7 +135,9 @@ func main() {
|
||||
"db": {
|
||||
"db": true,
|
||||
"session": true,
|
||||
"timeout": 2592000
|
||||
"timeout": 2592000,
|
||||
"history": false,
|
||||
"mode": "compatible"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -143,6 +145,27 @@ func main() {
|
||||
|
||||
缓存优先级: **Memory > Redis > DB**,自动穿透与回填
|
||||
|
||||
#### DB 缓存配置说明
|
||||
|
||||
| 配置项 | 默认值 | 说明 |
|
||||
|--------|--------|------|
|
||||
| `db` | false | 是否缓存数据库查询 |
|
||||
| `session` | true | 是否缓存 Session |
|
||||
| `timeout` | 2592000 | 过期时间(秒) |
|
||||
| `history` | false | 是否记录缓存历史,开启后每次新增/修改缓存都会记录到历史表 |
|
||||
| `mode` | compatible | 缓存表模式,见下表 |
|
||||
|
||||
**缓存表模式 (mode)**:
|
||||
|
||||
| 模式 | 说明 |
|
||||
|------|------|
|
||||
| `compatible` | **默认**。兼容模式:写新表,新表无数据时回退读老表;写入时自动删除老表同 key 记录;删除时同时删两表。适合从老版本平滑升级,老数据自然过期消亡 |
|
||||
| `new` | 只使用新表 `hotime_cache`,启动时自动迁移老表 `cached` 数据。老表保留由人工删除,不再被读写 |
|
||||
|
||||
> **升级建议**:从老版本升级时,建议先使用 `compatible` 模式运行一段时间(让老数据自然过期),确认无问题后再切换到 `new` 模式
|
||||
|
||||
> 从老版本升级时,建议使用 `compatible` 模式平滑过渡,待老表数据消亡后切换到 `new` 模式
|
||||
|
||||
### 错误码配置
|
||||
|
||||
```json
|
||||
@@ -342,6 +365,44 @@ data := that.Cache("key") // 获取
|
||||
that.Cache("key", nil) // 删除
|
||||
```
|
||||
|
||||
### 批量操作(性能优化)
|
||||
|
||||
当需要同时操作多个 Session 字段时,使用批量操作可显著提升性能:
|
||||
|
||||
```go
|
||||
// SessionsSet - 批量设置(N个字段只触发1次数据库写入)
|
||||
that.SessionsSet(Map{
|
||||
"user_id": userId,
|
||||
"username": "张三",
|
||||
"login_time": time.Now().Unix(),
|
||||
"role": "admin",
|
||||
})
|
||||
|
||||
// SessionsGet - 批量获取(1次调用获取多个字段)
|
||||
result := that.SessionsGet("user_id", "username", "role")
|
||||
// result = Map{"user_id": 123, "username": "张三", "role": "admin"}
|
||||
|
||||
userId := ObjToInt64(result["user_id"], nil)
|
||||
username := ObjToStr(result["username"])
|
||||
|
||||
// SessionsDelete - 批量删除(N个字段只触发1次数据库写入)
|
||||
that.SessionsDelete("token", "temp_code", "verify_expire")
|
||||
```
|
||||
|
||||
**性能对比**:
|
||||
|
||||
| 操作方式 | 设置10个字段 | 数据库写入次数 |
|
||||
|----------|-------------|---------------|
|
||||
| 逐个调用 `Session()` | 10次调用 | **10次** |
|
||||
| 使用 `SessionsSet()` | 1次调用 | **1次** |
|
||||
|
||||
| 操作方式 | 获取10个字段 | 缓存查询次数 |
|
||||
|----------|-------------|--------------|
|
||||
| 逐个调用 `Session()` | 10次调用 | **10次** |
|
||||
| 使用 `SessionsGet()` | 1次调用 | **1次** |
|
||||
|
||||
> 💡 **最佳实践**:当一次性操作 3 个以上字段时,建议使用批量操作
|
||||
|
||||
三级缓存自动运作:**Memory → Redis → Database**
|
||||
|
||||
## 数据库操作(简要)
|
||||
@@ -439,6 +500,8 @@ that.Log = Map{
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
. "code.hoteas.com/golang/hotime"
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
)
|
||||
@@ -481,14 +544,25 @@ func main() {
|
||||
return
|
||||
}
|
||||
|
||||
that.Session("user_id", user.GetInt64("id"))
|
||||
// 使用批量设置,一次写入多个字段
|
||||
that.SessionsSet(Map{
|
||||
"user_id": user.GetInt64("id"),
|
||||
"username": user.GetString("name"),
|
||||
"login_time": time.Now().Unix(),
|
||||
})
|
||||
that.Display(0, Map{"user": user})
|
||||
},
|
||||
|
||||
"info": func(that *Context) {
|
||||
userId := that.Session("user_id").ToInt64()
|
||||
// 使用批量获取,一次读取多个字段
|
||||
sess := that.SessionsGet("user_id", "username", "login_time")
|
||||
userId := ObjToInt64(sess["user_id"], nil)
|
||||
|
||||
user := that.Db.Get("user", "*", Map{"id": userId})
|
||||
that.Display(0, Map{"user": user})
|
||||
that.Display(0, Map{
|
||||
"user": user,
|
||||
"login_time": sess["login_time"],
|
||||
})
|
||||
},
|
||||
|
||||
"list": func(that *Context) {
|
||||
@@ -513,7 +587,8 @@ func main() {
|
||||
},
|
||||
|
||||
"logout": func(that *Context) {
|
||||
that.Session("user_id", nil)
|
||||
// 使用批量删除,一次清除多个字段
|
||||
that.SessionsDelete("user_id", "username", "login_time")
|
||||
that.Display(0, "退出成功")
|
||||
},
|
||||
},
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# HoTime 改进规划
|
||||
|
||||
本文档记录 HoTime 框架的待改进项和设计思考,供后续版本迭代参考。
|
||||
|
||||
---
|
||||
|
||||
## 一、备注语法优化
|
||||
|
||||
### 现状
|
||||
|
||||
当前字段备注语法使用空格、冒号、大括号组合:
|
||||
|
||||
```sql
|
||||
`status` int COMMENT '状态:0-正常,1-异常 这是数据库备注{这是前端提示}'
|
||||
```
|
||||
|
||||
### 问题
|
||||
|
||||
- 多种分隔符混用,不够直观
|
||||
- 显示名称如需包含特殊字符可能冲突
|
||||
|
||||
### 改进方向
|
||||
|
||||
使用 `|` 作为统一分隔符,保持干净清爽:
|
||||
|
||||
```sql
|
||||
-- 方案 A:简洁直观
|
||||
`status` int COMMENT '状态|0-正常,1-异常|请选择状态'
|
||||
-- 解析:显示名称|选项|提示
|
||||
|
||||
-- 方案 B:保持兼容,空格后内容仍作为数据库备注
|
||||
`status` int COMMENT '状态|0-正常,1-异常|请选择状态 这是数据库备注'
|
||||
```
|
||||
|
||||
### 待确认
|
||||
|
||||
- [ ] 是否需要保留空格后的数据库备注功能
|
||||
- [ ] 如果只有提示没有选项,如何表示(如 `名称||请输入名称`)
|
||||
- [ ] 是否向后兼容旧语法
|
||||
|
||||
---
|
||||
|
||||
## 二、SQLite 备注支持
|
||||
|
||||
### 现状
|
||||
|
||||
SQLite 不支持表/字段备注,需要手动在配置文件中设置。
|
||||
|
||||
### 问题
|
||||
|
||||
SQLite 项目配置工作量大,不够"快速开发"。
|
||||
|
||||
### 待探索方案
|
||||
|
||||
1. **配置文件方案(当前)**:通过 admin.json 手动配置,学习成本可接受
|
||||
2. **轻量级方案**:考虑是否有更简单的替代方案,但不增加复杂度
|
||||
|
||||
### 暂时结论
|
||||
|
||||
当前方案虽不完美,但符合"简单好用"原则,暂不改动。
|
||||
|
||||
---
|
||||
|
||||
## 三、软删除支持
|
||||
|
||||
### 现状
|
||||
|
||||
框架暂不支持软删除。
|
||||
|
||||
### 设计考量
|
||||
|
||||
软删除存在以下问题:
|
||||
- 数据安全性:软删除的数据仍可被恢复或误用
|
||||
- 查询复杂度:所有查询需要额外过滤条件
|
||||
- 存储膨胀:删除的数据持续占用空间
|
||||
|
||||
### 待探索方案
|
||||
|
||||
1. **可选软删除**:通过配置决定某张表是否启用软删除
|
||||
2. **日志表方案**:独立的操作日志表,只写不删不改
|
||||
- 记录所有增删改操作
|
||||
- 原表正常物理删除
|
||||
- 需要时可从日志恢复
|
||||
3. **归档表方案**:删除时移动到归档表
|
||||
|
||||
### 倾向方案
|
||||
|
||||
日志表方案更符合数据安全和审计需求:
|
||||
|
||||
```sql
|
||||
CREATE TABLE `_operation_log` (
|
||||
`id` int AUTO_INCREMENT,
|
||||
`table_name` varchar(100) COMMENT '操作表名',
|
||||
`record_id` int COMMENT '记录ID',
|
||||
`operation` varchar(20) COMMENT '操作类型:insert,update,delete',
|
||||
`old_data` text COMMENT '操作前数据(JSON)',
|
||||
`new_data` text COMMENT '操作后数据(JSON)',
|
||||
`operator_id` int COMMENT '操作人ID',
|
||||
`operator_table` varchar(100) COMMENT '操作人表名',
|
||||
`create_time` datetime COMMENT '操作时间',
|
||||
PRIMARY KEY (`id`)
|
||||
) COMMENT='操作日志';
|
||||
```
|
||||
|
||||
### 待确认
|
||||
|
||||
- [ ] 是否所有表都记录日志
|
||||
- [ ] 日志保留策略(永久/定期归档)
|
||||
- [ ] 是否提供恢复接口
|
||||
|
||||
---
|
||||
|
||||
## 四、多对多关联表增强
|
||||
|
||||
### 现状
|
||||
|
||||
关联表(如 `user_role`)按普通表处理,生成标准 CRUD。
|
||||
|
||||
### 可能的增强
|
||||
|
||||
1. **自动识别**:只有两个 `_id` 外键的表识别为关联表
|
||||
2. **专用接口**:生成关联管理接口(批量绑定/解绑)
|
||||
3. **级联查询**:自动生成带关联数据的查询
|
||||
|
||||
### 待确认
|
||||
|
||||
- [ ] 是否需要此功能
|
||||
- [ ] 如何保持简单性
|
||||
|
||||
---
|
||||
|
||||
## 五、自动填充字段扩展
|
||||
|
||||
### 现状
|
||||
|
||||
自动填充:
|
||||
- `create_time`:新增时自动填充当前时间
|
||||
- `modify_time`:新增/编辑时自动填充当前时间
|
||||
|
||||
### 可能的扩展
|
||||
|
||||
| 字段 | 填充时机 | 填充内容 |
|
||||
|------|----------|----------|
|
||||
| create_by | 新增 | 当前用户ID |
|
||||
| modify_by | 新增/编辑 | 当前用户ID |
|
||||
| create_ip | 新增 | 客户端IP |
|
||||
|
||||
### 待确认
|
||||
|
||||
- [ ] 是否需要扩展
|
||||
- [ ] 字段命名规范
|
||||
|
||||
---
|
||||
|
||||
## 六、版本控制/乐观锁
|
||||
|
||||
### 现状
|
||||
|
||||
`version` 字段规则存在但未启用。
|
||||
|
||||
### 待确认
|
||||
|
||||
- [ ] 是否需要支持乐观锁
|
||||
- [ ] 如果不需要,是否从默认规则中移除
|
||||
|
||||
---
|
||||
|
||||
## 改进优先级
|
||||
|
||||
| 优先级 | 改进项 | 状态 |
|
||||
|--------|--------|------|
|
||||
| 高 | 备注语法优化(`\|` 分隔符) | 待设计 |
|
||||
| 中 | 软删除/日志表支持 | 待设计 |
|
||||
| 低 | 多对多关联表增强 | 待评估 |
|
||||
| 低 | 自动填充字段扩展 | 待评估 |
|
||||
|
||||
---
|
||||
|
||||
## 更新记录
|
||||
|
||||
| 日期 | 内容 |
|
||||
|------|------|
|
||||
| 2026-01-24 | 初始版本,记录分析结果和待改进项 |
|
||||
@@ -0,0 +1,487 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"code.hoteas.com/golang/hotime"
|
||||
"code.hoteas.com/golang/hotime/cache"
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
)
|
||||
|
||||
const debugLogPath = `d:\work\hotimev1.5\.cursor\debug.log`
|
||||
|
||||
// debugLog 写入调试日志
|
||||
func debugLog(hypothesisId, location, message string, data map[string]interface{}) {
|
||||
logFile, _ := os.OpenFile(debugLogPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{
|
||||
"sessionId": "batch-cache-test",
|
||||
"runId": "test-run",
|
||||
"hypothesisId": hypothesisId,
|
||||
"location": location,
|
||||
"message": message,
|
||||
"data": data,
|
||||
"timestamp": time.Now().UnixMilli(),
|
||||
})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// TestBatchCacheOperations 测试所有批量缓存操作
|
||||
func TestBatchCacheOperations(app *hotime.Application) {
|
||||
fmt.Println("\n========== 批量缓存操作测试开始 ==========")
|
||||
|
||||
// 测试1: 测试 CacheMemory 批量操作
|
||||
fmt.Println("\n--- 测试1: CacheMemory 批量操作 ---")
|
||||
testCacheMemoryBatch(app)
|
||||
|
||||
// 测试2: 测试 CacheDb 批量操作
|
||||
fmt.Println("\n--- 测试2: CacheDb 批量操作 ---")
|
||||
testCacheDbBatch(app)
|
||||
|
||||
// 测试3: 测试 HoTimeCache 三级缓存批量操作
|
||||
fmt.Println("\n--- 测试3: HoTimeCache 三级缓存批量操作 ---")
|
||||
testHoTimeCacheBatch(app)
|
||||
|
||||
// 测试4: 测试 SessionIns 批量操作
|
||||
fmt.Println("\n--- 测试4: SessionIns 批量操作 ---")
|
||||
testSessionInsBatch(app)
|
||||
|
||||
// 测试5: 测试缓存反哺机制
|
||||
fmt.Println("\n--- 测试5: 缓存反哺机制测试 ---")
|
||||
testCacheBackfill(app)
|
||||
|
||||
// 测试6: 测试批量操作效率(一次性写入验证)
|
||||
fmt.Println("\n--- 测试6: 批量操作效率测试 ---")
|
||||
testBatchEfficiency(app)
|
||||
|
||||
fmt.Println("\n========== 批量缓存操作测试完成 ==========")
|
||||
}
|
||||
|
||||
// testCacheMemoryBatch 测试内存缓存批量操作
|
||||
func testCacheMemoryBatch(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheMemoryBatch:start", "开始测试CacheMemory批量操作", nil)
|
||||
// #endregion
|
||||
|
||||
memCache := &cache.CacheMemory{TimeOut: 3600, DbSet: true, SessionSet: true}
|
||||
memCache.SetError(&Error{})
|
||||
|
||||
// 测试 CachesSet
|
||||
testData := Map{
|
||||
"mem_key1": "value1",
|
||||
"mem_key2": "value2",
|
||||
"mem_key3": "value3",
|
||||
}
|
||||
memCache.CachesSet(testData)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheMemoryBatch:afterSet", "CacheMemory.CachesSet完成", map[string]interface{}{"count": len(testData)})
|
||||
// #endregion
|
||||
|
||||
// 测试 CachesGet
|
||||
keys := []string{"mem_key1", "mem_key2", "mem_key3", "mem_key_not_exist"}
|
||||
result := memCache.CachesGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheMemoryBatch:afterGet", "CacheMemory.CachesGet完成", map[string]interface{}{
|
||||
"requested_keys": keys,
|
||||
"result_count": len(result),
|
||||
"result_keys": getMapKeys(result),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result) != 3 {
|
||||
fmt.Printf(" [FAIL] CacheMemory.CachesGet: 期望3个结果,实际%d个\n", len(result))
|
||||
} else {
|
||||
fmt.Println(" [PASS] CacheMemory.CachesGet: 批量获取正确")
|
||||
}
|
||||
|
||||
// 测试 CachesDelete
|
||||
memCache.CachesDelete([]string{"mem_key1", "mem_key2"})
|
||||
result2 := memCache.CachesGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheMemoryBatch:afterDelete", "CacheMemory.CachesDelete完成", map[string]interface{}{
|
||||
"deleted_keys": []string{"mem_key1", "mem_key2"},
|
||||
"remaining_count": len(result2),
|
||||
"remaining_keys": getMapKeys(result2),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result2) != 1 || result2["mem_key3"] == nil {
|
||||
fmt.Printf(" [FAIL] CacheMemory.CachesDelete: 删除后期望1个结果,实际%d个\n", len(result2))
|
||||
} else {
|
||||
fmt.Println(" [PASS] CacheMemory.CachesDelete: 批量删除正确")
|
||||
}
|
||||
}
|
||||
|
||||
// testCacheDbBatch 测试数据库缓存批量操作
|
||||
func testCacheDbBatch(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheDbBatch:start", "开始测试CacheDb批量操作", nil)
|
||||
// #endregion
|
||||
|
||||
// 使用应用的数据库连接
|
||||
dbCache := &cache.CacheDb{
|
||||
TimeOut: 3600,
|
||||
DbSet: true,
|
||||
SessionSet: true,
|
||||
Mode: cache.CacheModeNew,
|
||||
Db: &app.Db,
|
||||
}
|
||||
dbCache.SetError(&Error{})
|
||||
|
||||
// 清理测试数据
|
||||
dbCache.CachesDelete([]string{"db_batch_key1", "db_batch_key2", "db_batch_key3"})
|
||||
|
||||
// 测试 CachesSet
|
||||
testData := Map{
|
||||
"db_batch_key1": "db_value1",
|
||||
"db_batch_key2": "db_value2",
|
||||
"db_batch_key3": "db_value3",
|
||||
}
|
||||
dbCache.CachesSet(testData)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheDbBatch:afterSet", "CacheDb.CachesSet完成", map[string]interface{}{"count": len(testData)})
|
||||
// #endregion
|
||||
|
||||
// 测试 CachesGet
|
||||
keys := []string{"db_batch_key1", "db_batch_key2", "db_batch_key3", "db_not_exist"}
|
||||
result := dbCache.CachesGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheDbBatch:afterGet", "CacheDb.CachesGet完成", map[string]interface{}{
|
||||
"requested_keys": keys,
|
||||
"result_count": len(result),
|
||||
"result_keys": getMapKeys(result),
|
||||
"result_values": result,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result) != 3 {
|
||||
fmt.Printf(" [FAIL] CacheDb.CachesGet: 期望3个结果,实际%d个\n", len(result))
|
||||
} else {
|
||||
fmt.Println(" [PASS] CacheDb.CachesGet: 批量获取正确")
|
||||
}
|
||||
|
||||
// 验证值正确性
|
||||
if result["db_batch_key1"] != "db_value1" {
|
||||
fmt.Printf(" [FAIL] CacheDb.CachesGet: db_batch_key1 值不正确,期望 db_value1,实际 %v\n", result["db_batch_key1"])
|
||||
} else {
|
||||
fmt.Println(" [PASS] CacheDb.CachesGet: 值内容正确")
|
||||
}
|
||||
|
||||
// 测试 CachesDelete
|
||||
dbCache.CachesDelete([]string{"db_batch_key1", "db_batch_key2"})
|
||||
result2 := dbCache.CachesGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("E", "batch_cache_test.go:testCacheDbBatch:afterDelete", "CacheDb.CachesDelete完成", map[string]interface{}{
|
||||
"deleted_keys": []string{"db_batch_key1", "db_batch_key2"},
|
||||
"remaining_count": len(result2),
|
||||
"remaining_keys": getMapKeys(result2),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result2) != 1 {
|
||||
fmt.Printf(" [FAIL] CacheDb.CachesDelete: 删除后期望1个结果,实际%d个\n", len(result2))
|
||||
} else {
|
||||
fmt.Println(" [PASS] CacheDb.CachesDelete: 批量删除正确")
|
||||
}
|
||||
|
||||
// 清理
|
||||
dbCache.CachesDelete([]string{"db_batch_key3"})
|
||||
}
|
||||
|
||||
// testHoTimeCacheBatch 测试 HoTimeCache 三级缓存批量操作
|
||||
func testHoTimeCacheBatch(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testHoTimeCacheBatch:start", "开始测试HoTimeCache三级缓存批量操作", nil)
|
||||
// #endregion
|
||||
|
||||
htCache := app.HoTimeCache
|
||||
|
||||
// 清理测试数据
|
||||
htCache.SessionsDelete([]string{
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key1",
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key2",
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key3",
|
||||
})
|
||||
|
||||
// 测试 SessionsSet
|
||||
testData := Map{
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key1": Map{"user": "test1", "role": "admin"},
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key2": Map{"user": "test2", "role": "user"},
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key3": Map{"user": "test3", "role": "guest"},
|
||||
}
|
||||
htCache.SessionsSet(testData)
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testHoTimeCacheBatch:afterSet", "HoTimeCache.SessionsSet完成", map[string]interface{}{"count": len(testData)})
|
||||
// #endregion
|
||||
|
||||
// 测试 SessionsGet
|
||||
keys := []string{
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key1",
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key2",
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key3",
|
||||
hotime.HEAD_SESSION_ADD + "ht_not_exist",
|
||||
}
|
||||
result := htCache.SessionsGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testHoTimeCacheBatch:afterGet", "HoTimeCache.SessionsGet完成", map[string]interface{}{
|
||||
"requested_keys": len(keys),
|
||||
"result_count": len(result),
|
||||
"result_keys": getMapKeys(result),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result) != 3 {
|
||||
fmt.Printf(" [FAIL] HoTimeCache.SessionsGet: 期望3个结果,实际%d个\n", len(result))
|
||||
} else {
|
||||
fmt.Println(" [PASS] HoTimeCache.SessionsGet: 批量获取正确")
|
||||
}
|
||||
|
||||
// 测试 SessionsDelete
|
||||
htCache.SessionsDelete([]string{
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key1",
|
||||
hotime.HEAD_SESSION_ADD + "ht_batch_key2",
|
||||
})
|
||||
result2 := htCache.SessionsGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testHoTimeCacheBatch:afterDelete", "HoTimeCache.SessionsDelete完成", map[string]interface{}{
|
||||
"remaining_count": len(result2),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result2) != 1 {
|
||||
fmt.Printf(" [FAIL] HoTimeCache.SessionsDelete: 删除后期望1个结果,实际%d个\n", len(result2))
|
||||
} else {
|
||||
fmt.Println(" [PASS] HoTimeCache.SessionsDelete: 批量删除正确")
|
||||
}
|
||||
|
||||
// 清理
|
||||
htCache.SessionsDelete([]string{hotime.HEAD_SESSION_ADD + "ht_batch_key3"})
|
||||
}
|
||||
|
||||
// testSessionInsBatch 测试 SessionIns 批量操作
|
||||
func testSessionInsBatch(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("B", "batch_cache_test.go:testSessionInsBatch:start", "开始测试SessionIns批量操作", nil)
|
||||
// #endregion
|
||||
|
||||
// 创建一个模拟的 SessionIns
|
||||
session := &hotime.SessionIns{
|
||||
SessionId: "test_batch_session_" + ObjToStr(time.Now().UnixNano()),
|
||||
}
|
||||
session.Init(app.HoTimeCache)
|
||||
|
||||
// 测试 SessionsSet
|
||||
testData := Map{
|
||||
"field1": "value1",
|
||||
"field2": 123,
|
||||
"field3": Map{"nested": "data"},
|
||||
}
|
||||
session.SessionsSet(testData)
|
||||
|
||||
// #region agent log
|
||||
debugLog("B", "batch_cache_test.go:testSessionInsBatch:afterSet", "SessionIns.SessionsSet完成", map[string]interface{}{
|
||||
"session_id": session.SessionId,
|
||||
"count": len(testData),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
// 测试 SessionsGet
|
||||
result := session.SessionsGet("field1", "field2", "field3", "not_exist")
|
||||
|
||||
// #region agent log
|
||||
debugLog("B", "batch_cache_test.go:testSessionInsBatch:afterGet", "SessionIns.SessionsGet完成", map[string]interface{}{
|
||||
"result_count": len(result),
|
||||
"result_keys": getMapKeys(result),
|
||||
"result": result,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result) != 3 {
|
||||
fmt.Printf(" [FAIL] SessionIns.SessionsGet: 期望3个结果,实际%d个\n", len(result))
|
||||
} else {
|
||||
fmt.Println(" [PASS] SessionIns.SessionsGet: 批量获取正确")
|
||||
}
|
||||
|
||||
// 验证值类型
|
||||
if result["field1"] != "value1" {
|
||||
fmt.Printf(" [FAIL] SessionIns.SessionsGet: field1 值不正确\n")
|
||||
} else {
|
||||
fmt.Println(" [PASS] SessionIns.SessionsGet: 字符串值正确")
|
||||
}
|
||||
|
||||
var convErr Error
|
||||
if ObjToInt(result["field2"], &convErr) != 123 {
|
||||
fmt.Printf(" [FAIL] SessionIns.SessionsGet: field2 值不正确\n")
|
||||
} else {
|
||||
fmt.Println(" [PASS] SessionIns.SessionsGet: 数值类型正确")
|
||||
}
|
||||
|
||||
// 测试 SessionsDelete
|
||||
session.SessionsDelete("field1", "field2")
|
||||
result2 := session.SessionsGet("field1", "field2", "field3")
|
||||
|
||||
// #region agent log
|
||||
debugLog("B", "batch_cache_test.go:testSessionInsBatch:afterDelete", "SessionIns.SessionsDelete完成", map[string]interface{}{
|
||||
"remaining_count": len(result2),
|
||||
"remaining_keys": getMapKeys(result2),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result2) != 1 {
|
||||
fmt.Printf(" [FAIL] SessionIns.SessionsDelete: 删除后期望1个结果,实际%d个\n", len(result2))
|
||||
} else {
|
||||
fmt.Println(" [PASS] SessionIns.SessionsDelete: 批量删除正确")
|
||||
}
|
||||
}
|
||||
|
||||
// testCacheBackfill 测试缓存反哺机制
|
||||
func testCacheBackfill(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testCacheBackfill:start", "开始测试缓存反哺机制", nil)
|
||||
// #endregion
|
||||
|
||||
htCache := app.HoTimeCache
|
||||
|
||||
// 直接写入数据库缓存(绕过 memory)模拟只有 db 有数据的情况
|
||||
dbCache := &cache.CacheDb{
|
||||
TimeOut: 3600,
|
||||
DbSet: true,
|
||||
SessionSet: true,
|
||||
Mode: cache.CacheModeNew,
|
||||
Db: &app.Db,
|
||||
}
|
||||
dbCache.SetError(&Error{})
|
||||
|
||||
testKey := "backfill_test_key_" + ObjToStr(time.Now().UnixNano())
|
||||
testValue := Map{"backfill": "test_data"}
|
||||
|
||||
// 直接写入 db
|
||||
dbCache.Cache(testKey, testValue)
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testCacheBackfill:dbWritten", "数据直接写入DB", map[string]interface{}{
|
||||
"key": testKey,
|
||||
"value": testValue,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
// 通过 HoTimeCache 批量获取,应该触发反哺到 memory
|
||||
keys := []string{testKey}
|
||||
result := htCache.CachesGet(keys)
|
||||
|
||||
// #region agent log
|
||||
debugLog("D", "batch_cache_test.go:testCacheBackfill:afterGet", "HoTimeCache.CachesGet完成", map[string]interface{}{
|
||||
"result_count": len(result),
|
||||
"has_key": result[testKey] != nil,
|
||||
})
|
||||
// #endregion
|
||||
|
||||
if len(result) != 1 || result[testKey] == nil {
|
||||
fmt.Println(" [FAIL] 缓存反哺: 从 DB 读取失败")
|
||||
} else {
|
||||
fmt.Println(" [PASS] 缓存反哺: 从 DB 读取成功")
|
||||
}
|
||||
|
||||
// 清理
|
||||
htCache.CachesDelete(keys)
|
||||
}
|
||||
|
||||
// testBatchEfficiency 测试批量操作效率
|
||||
func testBatchEfficiency(app *hotime.Application) {
|
||||
// #region agent log
|
||||
debugLog("A", "batch_cache_test.go:testBatchEfficiency:start", "开始测试批量操作效率", nil)
|
||||
// #endregion
|
||||
|
||||
session := &hotime.SessionIns{
|
||||
SessionId: "efficiency_test_" + ObjToStr(time.Now().UnixNano()),
|
||||
}
|
||||
session.Init(app.HoTimeCache)
|
||||
|
||||
// 记录批量设置开始时间
|
||||
startTime := time.Now()
|
||||
|
||||
// 设置10个字段
|
||||
testData := Map{}
|
||||
for i := 0; i < 10; i++ {
|
||||
testData[fmt.Sprintf("eff_field_%d", i)] = fmt.Sprintf("value_%d", i)
|
||||
}
|
||||
session.SessionsSet(testData)
|
||||
|
||||
batchDuration := time.Since(startTime)
|
||||
|
||||
// #region agent log
|
||||
debugLog("A", "batch_cache_test.go:testBatchEfficiency:batchSet", "批量设置完成", map[string]interface{}{
|
||||
"count": len(testData),
|
||||
"duration_ms": batchDuration.Milliseconds(),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
// 对比单个设置
|
||||
session2 := &hotime.SessionIns{
|
||||
SessionId: "efficiency_test_single_" + ObjToStr(time.Now().UnixNano()),
|
||||
}
|
||||
session2.Init(app.HoTimeCache)
|
||||
|
||||
startTime2 := time.Now()
|
||||
for i := 0; i < 10; i++ {
|
||||
session2.Session(fmt.Sprintf("single_field_%d", i), fmt.Sprintf("value_%d", i))
|
||||
}
|
||||
singleDuration := time.Since(startTime2)
|
||||
|
||||
// #region agent log
|
||||
debugLog("A", "batch_cache_test.go:testBatchEfficiency:singleSet", "单个设置完成", map[string]interface{}{
|
||||
"count": 10,
|
||||
"duration_ms": singleDuration.Milliseconds(),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
fmt.Printf(" 批量设置10个字段耗时: %v\n", batchDuration)
|
||||
fmt.Printf(" 单个设置10个字段耗时: %v\n", singleDuration)
|
||||
|
||||
if batchDuration < singleDuration {
|
||||
fmt.Println(" [PASS] 批量操作效率: 批量操作更快")
|
||||
} else {
|
||||
fmt.Println(" [WARN] 批量操作效率: 批量操作未体现优势(可能数据量太小)")
|
||||
}
|
||||
|
||||
// 批量获取测试
|
||||
startTime3 := time.Now()
|
||||
keys := make([]string, 10)
|
||||
for i := 0; i < 10; i++ {
|
||||
keys[i] = fmt.Sprintf("eff_field_%d", i)
|
||||
}
|
||||
session.SessionsGet(keys...)
|
||||
batchGetDuration := time.Since(startTime3)
|
||||
|
||||
// #region agent log
|
||||
debugLog("A", "batch_cache_test.go:testBatchEfficiency:batchGet", "批量获取完成", map[string]interface{}{
|
||||
"count": 10,
|
||||
"duration_ms": batchGetDuration.Milliseconds(),
|
||||
})
|
||||
// #endregion
|
||||
|
||||
fmt.Printf(" 批量获取10个字段耗时: %v\n", batchGetDuration)
|
||||
}
|
||||
|
||||
// getMapKeys 获取 Map 的所有键
|
||||
func getMapKeys(m Map) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
+404
-1
@@ -1,10 +1,12 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "code.hoteas.com/golang/hotime"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
. "code.hoteas.com/golang/hotime"
|
||||
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
. "code.hoteas.com/golang/hotime/db"
|
||||
)
|
||||
@@ -90,6 +92,16 @@ func main() {
|
||||
"upsert": func(that *Context) { that.Display(0, testUpsert(that)) },
|
||||
"transaction": func(that *Context) { that.Display(0, testTransaction(that)) },
|
||||
"rawsql": func(that *Context) { that.Display(0, testRawSQL(that)) },
|
||||
|
||||
// ==================== 缓存测试 ====================
|
||||
// 缓存全部测试
|
||||
"cache": func(that *Context) { that.Display(0, testCacheAll(that)) },
|
||||
"cache-compat": func(that *Context) { that.Display(0, testCacheCompatible(that)) },
|
||||
// 批量缓存操作测试
|
||||
"cache-batch": func(that *Context) {
|
||||
TestBatchCacheOperations(that.Application)
|
||||
that.Display(0, Map{"message": "批量缓存测试完成,请查看控制台输出和日志文件"})
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
@@ -903,3 +915,394 @@ func testRawSQL(that *Context) Map {
|
||||
result["success"] = true
|
||||
return result
|
||||
}
|
||||
|
||||
// ==================== 缓存测试 ====================
|
||||
func testCacheAll(that *Context) Map {
|
||||
result := Map{"name": "数据库缓存测试", "tests": Slice{}}
|
||||
tests := Slice{}
|
||||
|
||||
// 获取当前缓存模式
|
||||
cacheMode := "unknown"
|
||||
if that.Application.HoTimeCache != nil && that.Application.HoTimeCache.Config != nil {
|
||||
dbConfig := that.Application.HoTimeCache.Config.GetMap("db")
|
||||
if dbConfig != nil {
|
||||
cacheMode = dbConfig.GetString("mode")
|
||||
if cacheMode == "" {
|
||||
cacheMode = "new"
|
||||
}
|
||||
}
|
||||
}
|
||||
result["cache_mode"] = cacheMode
|
||||
|
||||
// 测试用的唯一前缀
|
||||
testPrefix := fmt.Sprintf("cache_test_%d_", time.Now().UnixNano())
|
||||
|
||||
// ==================== 1. 基础读写测试 ====================
|
||||
test1 := Map{"name": "1. 基础 set/get 测试"}
|
||||
testKey1 := testPrefix + "basic"
|
||||
testValue1 := Map{"name": "测试数据", "count": 123, "active": true}
|
||||
|
||||
// 设置缓存
|
||||
that.Application.Cache(testKey1, testValue1)
|
||||
|
||||
// 读取缓存
|
||||
cached1 := that.Application.Cache(testKey1)
|
||||
if cached1.Data != nil {
|
||||
cachedMap := cached1.ToMap()
|
||||
test1["result"] = cachedMap.GetString("name") == "测试数据" && cachedMap.GetInt("count") == 123
|
||||
test1["cached_value"] = cachedMap
|
||||
} else {
|
||||
test1["result"] = false
|
||||
test1["error"] = "缓存读取返回 nil"
|
||||
}
|
||||
tests = append(tests, test1)
|
||||
|
||||
// ==================== 2. 删除缓存测试 ====================
|
||||
test2 := Map{"name": "2. delete 删除缓存测试"}
|
||||
testKey2 := testPrefix + "delete"
|
||||
that.Application.Cache(testKey2, "删除测试值")
|
||||
|
||||
// 验证存在
|
||||
before := that.Application.Cache(testKey2)
|
||||
beforeExists := before.Data != nil
|
||||
|
||||
// 删除
|
||||
that.Application.Cache(testKey2, nil)
|
||||
|
||||
// 验证已删除
|
||||
after := that.Application.Cache(testKey2)
|
||||
afterExists := after.Data != nil
|
||||
|
||||
test2["result"] = beforeExists && !afterExists
|
||||
test2["before_exists"] = beforeExists
|
||||
test2["after_exists"] = afterExists
|
||||
tests = append(tests, test2)
|
||||
|
||||
// ==================== 3. 过期时间测试 ====================
|
||||
test3 := Map{"name": "3. 过期时间测试(短超时)"}
|
||||
testKey3 := testPrefix + "expire"
|
||||
|
||||
// 设置 2 秒过期
|
||||
that.Application.Cache(testKey3, "短期数据", 2)
|
||||
|
||||
// 立即读取应该存在
|
||||
immediate := that.Application.Cache(testKey3)
|
||||
immediateExists := immediate.Data != nil
|
||||
|
||||
test3["result"] = immediateExists
|
||||
test3["immediate_exists"] = immediateExists
|
||||
test3["note"] = "设置了2秒过期,可等待后再次访问验证过期"
|
||||
tests = append(tests, test3)
|
||||
|
||||
// ==================== 4. 不存在的 key 读取测试 ====================
|
||||
test4 := Map{"name": "4. 不存在的 key 读取测试"}
|
||||
nonExistKey := testPrefix + "non_exist_key_" + fmt.Sprintf("%d", time.Now().UnixNano())
|
||||
nonExist := that.Application.Cache(nonExistKey)
|
||||
test4["result"] = nonExist.Data == nil
|
||||
test4["value"] = nonExist.Data
|
||||
tests = append(tests, test4)
|
||||
|
||||
// ==================== 5. 重复 set 同一个 key 测试 ====================
|
||||
test5 := Map{"name": "5. 重复 set 同一个 key 测试"}
|
||||
testKey5 := testPrefix + "repeat"
|
||||
|
||||
that.Application.Cache(testKey5, "第一次值")
|
||||
first := that.Application.Cache(testKey5).ToStr()
|
||||
|
||||
that.Application.Cache(testKey5, "第二次值")
|
||||
second := that.Application.Cache(testKey5).ToStr()
|
||||
|
||||
that.Application.Cache(testKey5, Map{"version": 3})
|
||||
third := that.Application.Cache(testKey5).ToMap()
|
||||
|
||||
test5["result"] = first == "第一次值" && second == "第二次值" && third.GetInt("version") == 3
|
||||
test5["first"] = first
|
||||
test5["second"] = second
|
||||
test5["third"] = third
|
||||
tests = append(tests, test5)
|
||||
|
||||
// ==================== 6. 通配删除测试 ====================
|
||||
test6 := Map{"name": "6. 通配删除测试 (key*)"}
|
||||
wildcardPrefix := testPrefix + "wildcard_"
|
||||
|
||||
// 创建多个带相同前缀的缓存
|
||||
that.Application.Cache(wildcardPrefix+"a", "值A")
|
||||
that.Application.Cache(wildcardPrefix+"b", "值B")
|
||||
that.Application.Cache(wildcardPrefix+"c", "值C")
|
||||
|
||||
// 验证都存在
|
||||
aExists := that.Application.Cache(wildcardPrefix+"a").Data != nil
|
||||
bExists := that.Application.Cache(wildcardPrefix+"b").Data != nil
|
||||
cExists := that.Application.Cache(wildcardPrefix+"c").Data != nil
|
||||
allExistBefore := aExists && bExists && cExists
|
||||
|
||||
// 通配删除
|
||||
that.Application.Cache(wildcardPrefix+"*", nil)
|
||||
|
||||
// 验证都已删除
|
||||
aAfter := that.Application.Cache(wildcardPrefix+"a").Data != nil
|
||||
bAfter := that.Application.Cache(wildcardPrefix+"b").Data != nil
|
||||
cAfter := that.Application.Cache(wildcardPrefix+"c").Data != nil
|
||||
allDeletedAfter := !aAfter && !bAfter && !cAfter
|
||||
|
||||
test6["result"] = allExistBefore && allDeletedAfter
|
||||
test6["before"] = Map{"a": aExists, "b": bExists, "c": cExists}
|
||||
test6["after"] = Map{"a": aAfter, "b": bAfter, "c": cAfter}
|
||||
tests = append(tests, test6)
|
||||
|
||||
// ==================== 7. 不同数据类型测试 ====================
|
||||
test7 := Map{"name": "7. 不同数据类型存储测试"}
|
||||
|
||||
// 字符串
|
||||
that.Application.Cache(testPrefix+"type_string", "字符串值")
|
||||
typeString := that.Application.Cache(testPrefix + "type_string").ToStr()
|
||||
|
||||
// 整数
|
||||
that.Application.Cache(testPrefix+"type_int", 12345)
|
||||
typeInt := that.Application.Cache(testPrefix + "type_int").ToInt()
|
||||
|
||||
// 浮点数
|
||||
that.Application.Cache(testPrefix+"type_float", 3.14159)
|
||||
typeFloat := that.Application.Cache(testPrefix + "type_float").ToFloat64()
|
||||
|
||||
// 布尔值
|
||||
that.Application.Cache(testPrefix+"type_bool", true)
|
||||
typeBoolData := that.Application.Cache(testPrefix + "type_bool").Data
|
||||
typeBool := typeBoolData == true || typeBoolData == "true" || typeBoolData == 1.0
|
||||
|
||||
// Map
|
||||
that.Application.Cache(testPrefix+"type_map", Map{"key": "value", "num": 100})
|
||||
typeMap := that.Application.Cache(testPrefix + "type_map").ToMap()
|
||||
|
||||
// Slice
|
||||
that.Application.Cache(testPrefix+"type_slice", Slice{1, 2, 3, "four", Map{"five": 5}})
|
||||
typeSlice := that.Application.Cache(testPrefix + "type_slice").ToSlice()
|
||||
|
||||
test7["result"] = typeString == "字符串值" &&
|
||||
typeInt == 12345 &&
|
||||
typeFloat > 3.14 && typeFloat < 3.15 &&
|
||||
typeBool == true &&
|
||||
typeMap.GetString("key") == "value" &&
|
||||
len(typeSlice) == 5
|
||||
|
||||
test7["string"] = typeString
|
||||
test7["int"] = typeInt
|
||||
test7["float"] = typeFloat
|
||||
test7["bool"] = typeBool
|
||||
test7["map"] = typeMap
|
||||
test7["slice"] = typeSlice
|
||||
tests = append(tests, test7)
|
||||
|
||||
// ==================== 8. 自定义超时时间测试 ====================
|
||||
test8 := Map{"name": "8. 自定义超时时间参数测试"}
|
||||
testKey8 := testPrefix + "custom_timeout"
|
||||
|
||||
// 设置 3600 秒(1小时)过期
|
||||
that.Application.Cache(testKey8, "长期数据", 3600)
|
||||
longTerm := that.Application.Cache(testKey8)
|
||||
test8["result"] = longTerm.Data != nil
|
||||
test8["value"] = longTerm.ToStr()
|
||||
tests = append(tests, test8)
|
||||
|
||||
// ==================== 9. 查询缓存表状态 ====================
|
||||
test9 := Map{"name": "9. 缓存表状态查询"}
|
||||
|
||||
// 查询新表记录数
|
||||
prefix := that.Db.GetPrefix()
|
||||
newTableName := prefix + "hotime_cache"
|
||||
legacyTableName := prefix + "cached"
|
||||
|
||||
newCount := that.Db.Count(newTableName)
|
||||
test9["new_table_count"] = newCount
|
||||
test9["new_table_name"] = newTableName
|
||||
|
||||
// 尝试查询老表
|
||||
legacyCount := int64(-1)
|
||||
legacyExists := false
|
||||
legacyData := that.Db.Query("SELECT COUNT(*) as cnt FROM `" + legacyTableName + "`")
|
||||
if len(legacyData) > 0 {
|
||||
legacyExists = true
|
||||
legacyCount = legacyData[0].GetInt64("cnt")
|
||||
}
|
||||
test9["legacy_table_exists"] = legacyExists
|
||||
test9["legacy_table_count"] = legacyCount
|
||||
test9["legacy_table_name"] = legacyTableName
|
||||
|
||||
test9["result"] = newCount >= 0
|
||||
tests = append(tests, test9)
|
||||
|
||||
// ==================== 清理测试数据 ====================
|
||||
// 删除所有测试创建的缓存
|
||||
that.Application.Cache(testPrefix+"*", nil)
|
||||
|
||||
result["tests"] = tests
|
||||
result["success"] = true
|
||||
result["cleanup"] = "已清理所有测试缓存数据"
|
||||
return result
|
||||
}
|
||||
|
||||
// testCacheCompatible 专门测试兼容模式 - 白盒测试
|
||||
func testCacheCompatible(that *Context) Map {
|
||||
result := Map{
|
||||
"test_name": "兼容模式白盒测试",
|
||||
"timestamp": time.Now().Format("2006-01-02 15:04:05"),
|
||||
}
|
||||
|
||||
prefix := that.Db.GetPrefix()
|
||||
newTableName := prefix + "hotime_cache"
|
||||
legacyTableName := prefix + "cached"
|
||||
|
||||
tests := Slice{}
|
||||
|
||||
// ==================== 1. 查询当前模式 ====================
|
||||
test1 := Map{"name": "1. 查询当前缓存模式"}
|
||||
// 读取配置确认模式
|
||||
cacheConfig := that.Application.Config.GetMap("cache")
|
||||
dbConfig := cacheConfig.GetMap("db")
|
||||
mode := dbConfig.GetString("mode")
|
||||
if mode == "" {
|
||||
mode = "默认(compatible)"
|
||||
}
|
||||
test1["mode"] = mode
|
||||
test1["result"] = true
|
||||
tests = append(tests, test1)
|
||||
|
||||
// ==================== 2. 查询老表数据 ====================
|
||||
test2 := Map{"name": "2. 查询老表cached现有数据"}
|
||||
|
||||
legacyData := that.Db.Query("SELECT * FROM `" + legacyTableName + "` LIMIT 5")
|
||||
test2["legacy_table"] = legacyTableName
|
||||
test2["count"] = len(legacyData)
|
||||
test2["data"] = legacyData
|
||||
test2["result"] = true
|
||||
tests = append(tests, test2)
|
||||
|
||||
// ==================== 3. 查询新表数据 ====================
|
||||
test3 := Map{"name": "3. 查询新表hotime_cache现有数据"}
|
||||
|
||||
newData := that.Db.Query("SELECT * FROM `" + newTableName + "` LIMIT 5")
|
||||
test3["new_table"] = newTableName
|
||||
test3["count"] = len(newData)
|
||||
test3["data"] = newData
|
||||
test3["result"] = true
|
||||
tests = append(tests, test3)
|
||||
|
||||
// ==================== 4. 测试老表回退读取 ====================
|
||||
test4 := Map{"name": "4. 测试兼容模式老表回退读取"}
|
||||
|
||||
// 插入一条未过期的老表数据进行测试
|
||||
testKey4 := "test_compat_fallback_" + ObjToStr(time.Now().UnixNano())
|
||||
testValue4 := Map{"admin_id": 999, "admin_name": "测试老数据"}
|
||||
testValueJson4, _ := json.Marshal(Map{"data": testValue4})
|
||||
|
||||
// 在老表插入未过期数据
|
||||
that.Db.Insert(legacyTableName, Map{
|
||||
"key": testKey4,
|
||||
"value": string(testValueJson4),
|
||||
"endtime": time.Now().Unix() + 3600, // 1小时后过期
|
||||
"time": time.Now().UnixNano(),
|
||||
})
|
||||
|
||||
test4["test_key"] = testKey4
|
||||
|
||||
// 确保新表没有这个 key
|
||||
newExists := that.Db.Get(newTableName, "*", Map{"key": testKey4})
|
||||
test4["key_in_new_table"] = newExists != nil
|
||||
|
||||
// 通过缓存 API 读取(应该回退到老表)
|
||||
cacheValue := that.Application.Cache(testKey4)
|
||||
test4["cache_api_result"] = cacheValue.Data
|
||||
|
||||
// 直接从老表读取确认
|
||||
legacyValue := that.Db.Get(legacyTableName, "*", Map{"key": testKey4})
|
||||
if legacyValue != nil {
|
||||
test4["legacy_db_value"] = legacyValue.GetString("value")
|
||||
test4["legacy_db_endtime"] = legacyValue.GetInt64("endtime")
|
||||
test4["legacy_db_endtime_readable"] = time.Unix(legacyValue.GetInt64("endtime"), 0).Format("2006-01-02 15:04:05")
|
||||
}
|
||||
|
||||
// 验证:新表没数据,但缓存API能读到老表数据
|
||||
test4["result"] = newExists == nil && cacheValue.Data != nil
|
||||
tests = append(tests, test4)
|
||||
|
||||
// ==================== 5. 测试写新删老 ====================
|
||||
test5 := Map{"name": "5. 测试兼容模式写新删老"}
|
||||
|
||||
testKey5 := "test_compat_write_" + ObjToStr(time.Now().UnixNano())
|
||||
testValue5 := "兼容模式测试数据"
|
||||
|
||||
// 先在老表插入一条数据
|
||||
that.Db.Insert(legacyTableName, Map{
|
||||
"key": testKey5,
|
||||
"value": `{"data":"老表原始数据"}`,
|
||||
"endtime": time.Now().Unix() + 3600, // 1小时后过期
|
||||
"time": time.Now().UnixNano(),
|
||||
})
|
||||
|
||||
// 确认老表有数据
|
||||
legacyBefore := that.Db.Get(legacyTableName, "*", Map{"key": testKey5})
|
||||
test5["step1_legacy_before"] = legacyBefore != nil
|
||||
|
||||
// 通过缓存 API 写入(应该写新表并删老表)
|
||||
that.Application.Cache(testKey5, testValue5)
|
||||
|
||||
// 检查新表
|
||||
newAfter := that.Db.Get(newTableName, "*", Map{"key": testKey5})
|
||||
test5["step2_new_after"] = newAfter != nil
|
||||
if newAfter != nil {
|
||||
test5["new_value"] = newAfter.GetString("value")
|
||||
}
|
||||
|
||||
// 检查老表(应该被删除)
|
||||
legacyAfter := that.Db.Get(legacyTableName, "*", Map{"key": testKey5})
|
||||
test5["step3_legacy_after_deleted"] = legacyAfter == nil
|
||||
|
||||
test5["result"] = legacyBefore != nil && newAfter != nil && legacyAfter == nil
|
||||
tests = append(tests, test5)
|
||||
|
||||
// ==================== 6. 测试删除同时删两表 ====================
|
||||
test6 := Map{"name": "6. 测试兼容模式删除(删除两表)"}
|
||||
|
||||
testKey6 := "test_compat_delete_" + ObjToStr(time.Now().UnixNano())
|
||||
|
||||
// 在老表插入
|
||||
that.Db.Insert(legacyTableName, Map{
|
||||
"key": testKey6,
|
||||
"value": `{"data":"待删除老数据"}`,
|
||||
"endtime": time.Now().Unix() + 3600,
|
||||
"time": time.Now().UnixNano(),
|
||||
})
|
||||
|
||||
// 在新表插入
|
||||
that.Db.Insert(newTableName, Map{
|
||||
"key": testKey6,
|
||||
"value": `"待删除新数据"`,
|
||||
"end_time": time.Now().Add(time.Hour).Format("2006-01-02 15:04:05"),
|
||||
"state": 0,
|
||||
"create_time": time.Now().Format("2006-01-02 15:04:05"),
|
||||
"modify_time": time.Now().Format("2006-01-02 15:04:05"),
|
||||
})
|
||||
|
||||
// 确认两表都有数据
|
||||
test6["before_legacy"] = that.Db.Get(legacyTableName, "*", Map{"key": testKey6}) != nil
|
||||
test6["before_new"] = that.Db.Get(newTableName, "*", Map{"key": testKey6}) != nil
|
||||
|
||||
// 通过缓存 API 删除
|
||||
that.Application.Cache(testKey6, nil)
|
||||
|
||||
// 确认两表都被删除
|
||||
test6["after_legacy_deleted"] = that.Db.Get(legacyTableName, "*", Map{"key": testKey6}) == nil
|
||||
test6["after_new_deleted"] = that.Db.Get(newTableName, "*", Map{"key": testKey6}) == nil
|
||||
|
||||
test6["result"] = test6.GetBool("before_legacy") && test6.GetBool("before_new") &&
|
||||
test6.GetBool("after_legacy_deleted") && test6.GetBool("after_new_deleted")
|
||||
tests = append(tests, test6)
|
||||
|
||||
// ==================== 7. 清理测试数据 ====================
|
||||
that.Db.Delete(newTableName, Map{"key[~]": "test_compat_%"})
|
||||
that.Db.Delete(legacyTableName, Map{"key[~]": "test_compat_%"})
|
||||
|
||||
result["tests"] = tests
|
||||
result["success"] = true
|
||||
return result
|
||||
}
|
||||
|
||||
@@ -18,6 +18,7 @@ github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
|
||||
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
|
||||
github.com/gomodule/redigo v1.8.5 h1:nRAxCa+SVsyjSBrtZmG/cqb6VbTmuRzpg/PoTFlpumc=
|
||||
github.com/gomodule/redigo v1.8.5/go.mod h1:P9dn9mFrCBvWhGE1wpxx6fgq7BAeLBk+UUUzlpkBYO0=
|
||||
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
|
||||
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 h1:2VTzZjLZBgl62/EtslCrtky5vbi9dd7HrQPQIx6wqiw=
|
||||
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
|
||||
@@ -56,6 +57,7 @@ github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.364 h1:X1Jw
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.364/go.mod h1:7sCQWVkxcsR38nffDW057DRGk8mUjK1Ing/EFOK8s8Y=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/ocr v1.0.364 h1:kbor60vo37v7Hu+i17gooox9Rw281fVHNna8zwtDG1w=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/ocr v1.0.364/go.mod h1:LeIUBOLhc+Y5YCEpZrULPD9lgoXXV4/EmIcoEvmHz9c=
|
||||
github.com/tidwall/pretty v1.0.0 h1:HsD+QiTn7sK6flMKIvNmpqz1qrpP3Ps6jOKIKMooyg4=
|
||||
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
|
||||
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
|
||||
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
|
||||
@@ -69,7 +71,6 @@ go.mongodb.org/mongo-driver v1.10.1 h1:NujsPveKwHaWuKUer/ceo9DzEe7HIj1SlJ6uvXZG0
|
||||
go.mongodb.org/mongo-driver v1.10.1/go.mod h1:z4XpeoU6w+9Vht+jAFyLgVrD+jGSQQe0+CBWFHNiHt8=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
|
||||
golang.org/x/crypto v0.0.0-20220331220935-ae2d96664a29 h1:tkVvjkPTB7pnW3jnid7kNyAMPVWllTNOf/qKDze4p9o=
|
||||
golang.org/x/crypto v0.0.0-20220331220935-ae2d96664a29/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
|
||||
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d h1:sK3txAijHtOK88l68nt020reeT1ZdKLIYetKl95FzVY=
|
||||
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
|
||||
@@ -95,6 +96,7 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
@@ -102,6 +104,6 @@ gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8
|
||||
gopkg.in/h2non/gock.v1 v1.0.15 h1:SzLqcIlb/fDfg7UvukMpNcWsu7sI5tWwL+KCATZqks0=
|
||||
gopkg.in/h2non/gock.v1 v1.0.15/go.mod h1:sX4zAkdYX1TRGJ2JY156cFspQn4yRWn6p9EMdODlynE=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
+92
-36
@@ -2,12 +2,13 @@ package log
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func GetLog(path string, showCodeLine bool) *log.Logger {
|
||||
@@ -73,41 +74,98 @@ func (that *MyHook) Fire(entry *log.Entry) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// 对caller进行递归查询, 直到找到非logrus包产生的第一个调用.
|
||||
// 因为filename我获取到了上层目录名, 因此所有logrus包的调用的文件名都是 logrus/...
|
||||
// 因此通过排除logrus开头的文件名, 就可以排除所有logrus包的自己的函数调用
|
||||
func findCaller(skip int) string {
|
||||
file := ""
|
||||
line := 0
|
||||
for i := 0; i < 10; i++ {
|
||||
file, line = getCaller(skip + i)
|
||||
if !strings.HasPrefix(file, "logrus") {
|
||||
j := 0
|
||||
for true {
|
||||
j++
|
||||
if file == "common/error.go" {
|
||||
file, line = getCaller(skip + i + j)
|
||||
}
|
||||
if file == "db/hotimedb.go" {
|
||||
file, line = getCaller(skip + i + j)
|
||||
}
|
||||
if file == "code/makecode.go" {
|
||||
file, line = getCaller(skip + i + j)
|
||||
}
|
||||
if strings.Index(file, "common/") == 0 {
|
||||
file, line = getCaller(skip + i + j)
|
||||
}
|
||||
if strings.Contains(file, "application.go") {
|
||||
file, line = getCaller(skip + i + j)
|
||||
}
|
||||
if j == 5 {
|
||||
break
|
||||
}
|
||||
}
|
||||
// 最大框架层数限制 - 超过这个层数后不再跳过,防止误过滤应用层
|
||||
const maxFrameworkDepth = 10
|
||||
|
||||
break
|
||||
// isHoTimeFrameworkFile 判断是否是 HoTime 框架文件
|
||||
// 更精确的匹配:只有明确属于框架的文件才会被跳过
|
||||
func isHoTimeFrameworkFile(file string) bool {
|
||||
// 1. logrus 日志库内部文件(支持带版本号的路径,如 logrus@v1.8.1/entry.go)
|
||||
if strings.HasPrefix(file, "logrus/") || strings.HasPrefix(file, "logrus@") {
|
||||
return true
|
||||
}
|
||||
|
||||
// 2. Go 运行时文件
|
||||
if strings.HasPrefix(file, "runtime/") {
|
||||
return true
|
||||
}
|
||||
|
||||
// 3. HoTime 框架核心文件 - 通过包含 "hotime" 或框架特有文件名来识别
|
||||
// 检查路径中是否包含 hotime 框架标识
|
||||
lowerFile := strings.ToLower(file)
|
||||
if strings.Contains(lowerFile, "hotime") {
|
||||
// 是 hotime 框架的一部分,检查是否是核心模块
|
||||
frameworkDirs := []string{"db/", "common/", "code/", "cache/", "log/", "dri/"}
|
||||
for _, dir := range frameworkDirs {
|
||||
if strings.Contains(file, dir) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
// 框架核心文件(在 hotime 根目录下的 .go 文件)
|
||||
if strings.HasSuffix(file, "application.go") ||
|
||||
strings.HasSuffix(file, "context.go") ||
|
||||
strings.HasSuffix(file, "session.go") ||
|
||||
strings.HasSuffix(file, "const.go") ||
|
||||
strings.HasSuffix(file, "type.go") ||
|
||||
strings.HasSuffix(file, "var.go") ||
|
||||
strings.HasSuffix(file, "mime.go") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// 4. 直接匹配框架核心目录(用于没有完整路径的情况)
|
||||
// 只匹配 "db/xxx.go" 这种在框架核心目录下的文件
|
||||
frameworkCoreDirs := []string{"db/", "common/", "code/", "cache/"}
|
||||
for _, dir := range frameworkCoreDirs {
|
||||
if strings.HasPrefix(file, dir) {
|
||||
// 额外检查:确保不是用户项目中同名目录
|
||||
// 框架文件通常有特定的文件名
|
||||
frameworkFiles := []string{
|
||||
"query.go", "crud.go", "where.go", "builder.go", "db.go",
|
||||
"dialect.go", "aggregate.go", "transaction.go", "identifier.go",
|
||||
"error.go", "func.go", "map.go", "obj.go", "slice.go",
|
||||
"makecode.go", "template.go", "config.go",
|
||||
"cache.go", "cache_db.go", "cache_memory.go", "cache_redis.go",
|
||||
}
|
||||
for _, f := range frameworkFiles {
|
||||
if strings.HasSuffix(file, f) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// 对caller进行递归查询, 直到找到非框架层产生的第一个调用.
|
||||
// 遍历调用栈,跳过框架层文件,找到应用层代码
|
||||
// 使用层数限制确保不会误过滤应用层同名目录
|
||||
func findCaller(skip int) string {
|
||||
frameworkCount := 0 // 连续框架层计数
|
||||
|
||||
// 遍历调用栈,找到第一个非框架文件
|
||||
for i := 0; i < 20; i++ {
|
||||
file, line := getCaller(skip + i)
|
||||
if file == "" {
|
||||
break
|
||||
}
|
||||
|
||||
if isHoTimeFrameworkFile(file) {
|
||||
frameworkCount++
|
||||
// 层数限制:如果已经跳过太多层,停止跳过
|
||||
if frameworkCount >= maxFrameworkDepth {
|
||||
return fmt.Sprintf("%s:%d", file, line)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// 找到非框架文件,返回应用层代码位置
|
||||
return fmt.Sprintf("%s:%d", file, line)
|
||||
}
|
||||
|
||||
// 如果找不到应用层,返回最初的调用者
|
||||
file, line := getCaller(skip)
|
||||
return fmt.Sprintf("%s:%d", file, line)
|
||||
}
|
||||
|
||||
@@ -117,14 +175,12 @@ func findCaller(skip int) string {
|
||||
// 因为文件的全路径往往很长, 而文件名在多个包中往往有重复, 因此这里选择多取一层, 取到文件所在的上层目录那层.
|
||||
func getCaller(skip int) (string, int) {
|
||||
_, file, line, ok := runtime.Caller(skip)
|
||||
//fmt.Println(file)
|
||||
//fmt.Println(line)
|
||||
if !ok {
|
||||
return "", 0
|
||||
}
|
||||
n := 0
|
||||
for i := len(file) - 1; i > 0; i-- {
|
||||
if file[i] == '/' {
|
||||
if file[i] == '/' || file[i] == '\\' { // 同时处理 / 和 \ 路径分隔符
|
||||
n++
|
||||
if n >= 2 {
|
||||
file = file[i+1:]
|
||||
|
||||
+120
-1
@@ -1,9 +1,13 @@
|
||||
package hotime
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
. "code.hoteas.com/golang/hotime/cache"
|
||||
. "code.hoteas.com/golang/hotime/common"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// session对象
|
||||
@@ -17,6 +21,15 @@ type SessionIns struct {
|
||||
|
||||
// set 保存 session 到缓存,必须在锁内调用或传入深拷贝的 map
|
||||
func (that *SessionIns) setWithCopy() {
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(`d:\work\hotimev1.5\.cursor\debug.log`, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{"sessionId": "debug-session", "runId": "run1", "hypothesisId": "A", "location": "session.go:setWithCopy", "message": "Session写入数据库触发", "data": map[string]interface{}{"session_id": that.SessionId, "map_size": len(that.Map)}, "timestamp": time.Now().UnixMilli()})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
|
||||
// 深拷贝 Map 防止并发修改
|
||||
that.mutex.RLock()
|
||||
copyMap := make(Map, len(that.Map))
|
||||
@@ -36,6 +49,15 @@ func (that *SessionIns) Session(key string, data ...interface{}) *Obj {
|
||||
that.mutex.Unlock()
|
||||
|
||||
if len(data) != 0 {
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(`d:\work\hotimev1.5\.cursor\debug.log`, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{"sessionId": "debug-session", "runId": "run1", "hypothesisId": "B", "location": "session.go:Session", "message": "Session.Set调用", "data": map[string]interface{}{"key": key, "is_delete": data[0] == nil}, "timestamp": time.Now().UnixMilli()})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
|
||||
that.mutex.Lock()
|
||||
if data[0] == nil {
|
||||
delete(that.Map, key)
|
||||
@@ -55,6 +77,103 @@ func (that *SessionIns) Session(key string, data ...interface{}) *Obj {
|
||||
return result
|
||||
}
|
||||
|
||||
// SessionsSet 批量设置session字段,只触发一次数据库写入
|
||||
// 用法:that.SessionsSet(Map{"key1": value1, "key2": value2, ...})
|
||||
// 性能优化:设置N个字段只触发1次数据库写入(而非N次)
|
||||
func (that *SessionIns) SessionsSet(data Map) {
|
||||
if len(data) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(`d:\work\hotimev1.5\.cursor\debug.log`, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
keys := make([]string, 0, len(data))
|
||||
for k := range data {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{"sessionId": "debug-session", "runId": "run1", "hypothesisId": "C", "location": "session.go:SessionsSet", "message": "SessionsSet批量设置", "data": map[string]interface{}{"keys": keys, "count": len(data)}, "timestamp": time.Now().UnixMilli()})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
|
||||
that.mutex.Lock()
|
||||
if that.Map == nil {
|
||||
that.getWithoutLock()
|
||||
}
|
||||
|
||||
// 批量设置所有字段
|
||||
for key, value := range data {
|
||||
if value == nil {
|
||||
delete(that.Map, key)
|
||||
} else {
|
||||
that.Map[key] = value
|
||||
}
|
||||
}
|
||||
that.mutex.Unlock()
|
||||
|
||||
// 只触发一次数据库写入
|
||||
that.setWithCopy()
|
||||
}
|
||||
|
||||
// SessionsDelete 批量删除session字段,只触发一次数据库写入
|
||||
// 用法:that.SessionsDelete("key1", "key2", ...)
|
||||
func (that *SessionIns) SessionsDelete(keys ...string) {
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// #region agent log
|
||||
logFile, _ := os.OpenFile(`d:\work\hotimev1.5\.cursor\debug.log`, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if logFile != nil {
|
||||
logEntry, _ := json.Marshal(map[string]interface{}{"sessionId": "debug-session", "runId": "run1", "hypothesisId": "C", "location": "session.go:SessionsDelete", "message": "SessionsDelete批量删除", "data": map[string]interface{}{"keys": keys, "count": len(keys)}, "timestamp": time.Now().UnixMilli()})
|
||||
logFile.Write(append(logEntry, '\n'))
|
||||
logFile.Close()
|
||||
}
|
||||
// #endregion
|
||||
|
||||
that.mutex.Lock()
|
||||
if that.Map == nil {
|
||||
that.getWithoutLock()
|
||||
}
|
||||
|
||||
// 批量删除所有字段
|
||||
for _, key := range keys {
|
||||
delete(that.Map, key)
|
||||
}
|
||||
that.mutex.Unlock()
|
||||
|
||||
// 只触发一次数据库写入
|
||||
that.setWithCopy()
|
||||
}
|
||||
|
||||
// SessionsGet 批量获取session字段
|
||||
// 用法:result := that.SessionsGet("key1", "key2", ...)
|
||||
// 返回 Map,key 为字段名,value 为字段值(不存在的 key 不包含在结果中)
|
||||
func (that *SessionIns) SessionsGet(keys ...string) Map {
|
||||
if len(keys) == 0 {
|
||||
return Map{}
|
||||
}
|
||||
|
||||
that.mutex.Lock()
|
||||
if that.Map == nil {
|
||||
that.getWithoutLock()
|
||||
}
|
||||
that.mutex.Unlock()
|
||||
|
||||
result := make(Map, len(keys))
|
||||
that.mutex.RLock()
|
||||
for _, key := range keys {
|
||||
if value, exists := that.Map[key]; exists {
|
||||
result[key] = value
|
||||
}
|
||||
}
|
||||
that.mutex.RUnlock()
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// getWithoutLock 内部使用,调用前需要已持有锁
|
||||
func (that *SessionIns) getWithoutLock() {
|
||||
that.Map = that.HoTimeCache.Session(HEAD_SESSION_ADD + that.SessionId).ToMap()
|
||||
|
||||
@@ -107,6 +107,8 @@ var ConfigNote = Map{
|
||||
"timeout": "默认60 * 60 * 24 * 30,非必须,过期时间,超时自动删除",
|
||||
"db": "默认false,非必须,缓存数据库,启用后能减少数据库的读写压力",
|
||||
"session": "默认true,非必须,缓存web session,同时缓存session保持的用户缓存",
|
||||
"history": "默认false,非必须,是否开启缓存历史记录,开启后每次新增/修改缓存都会记录到历史表,历史表一旦创建不会自动删除",
|
||||
"mode": "默认compatible,非必须,缓存表模式。compatible:兼容模式(写新表,新表无数据时回退读老表,写入时自动删除老表同key记录,删除时同时删两表,老数据自然过期消亡);new:只使用新表hotime_cache(自动迁移老表cached数据,老表保留由人工删除)",
|
||||
},
|
||||
"redis": Map{
|
||||
"host": "默认服务ip:127.0.0.1,必须,如果需要使用redis服务时配置,",
|
||||
|
||||
Reference in New Issue
Block a user