forgejo/modules
Victoria Nadasdi 54acfa8880
refactor: redis queue backend test cleanup
Summary:
- Move existing test under a `testify` Suite as `baseRedisWithServerTestSuite`
  - Those tests require real redis server.
- Add `go.uber.org/mock/mockgen@latest` as dependency
  - as a tool (Makefile).
  - in the `go.mod` file.
- Mock redis client lives under a `mock` directory under the queue module.
  - That mock module has an extra hand-written mock in-memory redis-like struct.
- Add tests using the mock redis client.
- Changed the logic around queue provider creation.
  - Now the `getNewQueue` returns a Queue provider directly, not an init
    function to create it.

The whole Queue module is close to impossible to test properly because
everything is private, everything goes through a struct route. Because
of that, we can't test for example what keys are used for given queue.

To overcome this, as a first step I removed one step from that hard
route by allowing custom calls to create new queue provider. To achieve
this, I moved the creation logic into the `getNewQueue` (previously it
was `getNewQueueFn`). That changes nothing on that side, everything goes
as before, except the `newXXX` call happens directly in that function
and not outside that.

That made it possible to add extra provider specific parameters to those
function (`newXXX`). For example a client on redis. Calling it through
the `getNewQueue` function, it gets `nil`.

- If the provided client is not `nil`, it will use that instead of the
connection string.
- If it's `nil` (default behaviour), it creates a new redis client as it
  did before, no changes to that.

The rest of the provider code is unchanged. All these changes were
required to make it possible to generate mock clients for providers and
use them.

For the tests, the existing two test cases are good with redis server,
and they need some extra helpers, for example to start a new redis
server if required, or waiting on a redis server to be ready to use.
These helpers are only required for test cases using real redis server.

For better isolation, moved existing test under a testify Suite, and
moved them into a new test file called `base_redis_with_server_test.go`
because, well they test the code with server. These tests do exactly the
same as before, calling the same sub-tests the same way as before, the
only change is the structure of the test (remove repetition, scope
server related helper functions).

Finally, we can create unit tests without redis server. The main focus of
this group of tests are higher level overview of operations. With the
mock redis client we can set up expectations about used queue names,
received values, return value to simulate faulty state.

These new unit test functions don't test all functionality, at least
it's not aimed for it now. It's more about the possibility of doing that
and add extra tests around parts we couldn't test before, for example
key.

What extra features can test the new unit test group:
- What is the received key for given queue? For example using `prefix`,
  or if all the `SXxx` calls are expected to use `queue_unique` if
  it's a unique queue.
- If it's not a unique queue, no `SXxx` functions are called, because
  those sets are used only to check if a value is unique or not.
- `HasItem` return `false` always if it's a non-unique queue.
- All functions are called exactly `N` times, and we don't have any
  unexpected calls to redis from the code.

Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
2024-05-21 18:02:33 +02:00
..
actions Resolve lint for unused parameter and unnecessary type arguments (#30750) 2024-05-05 08:38:16 +01:00
activitypub enhance test & fix reviews 2024-05-14 08:24:31 +02:00
analyze
assetfs
auth Remove external API calls in TestPassword (#30716) 2024-05-05 12:15:40 +01:00
avatar Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
base Refactor StringsToInt64s (#29967) 2024-03-26 19:04:26 +01:00
cache
charset s/Gitea/Forgejo in various log messages and comments 2024-04-21 21:26:15 +05:00
container Add container.FilterSlice function (gitea#30339) 2024-04-16 11:49:44 +02:00
csv Refactor locale&string&template related code (#29165) 2024-02-16 15:20:52 +01:00
emoji
eventsource
forgefed missed by diff 2024-05-14 08:48:05 +02:00
generate Port "Use general token signing secret" 2024-02-19 20:49:37 +01:00
git feat: wiki search using git-grep 2024-05-20 13:48:50 +00:00
gitgraph Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
gitrepo Move get/set default branch from git package to gitrepo package to hide repopath (#29126) 2024-03-11 23:36:59 +07:00
graceful [FIX] make pprof labels conformant with prometheus spec 2024-03-31 17:07:14 +02:00
hcaptcha
highlight Fix 2024-03-30 15:23:48 +01:00
hostmatcher
html
httpcache [BRANDING] add X-Forgejo-* headers 2024-02-05 16:02:14 +01:00
httplib Skip gzip for some well-known compressed file types (#30796) 2024-05-05 12:15:40 +01:00
indexer Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
issue/template Extend issue template yaml engine (#29274) 2024-03-06 12:10:47 +08:00
json
label
lfs fix: close file in the Upload func (#30262) 2024-04-07 15:40:31 +02:00
log Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
markup Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
mcaptcha
metrics
migration Refactor locale&string&template related code (#29165) 2024-02-16 15:20:52 +01:00
nosql s/Gitea/Forgejo in various log messages and comments 2024-04-21 21:26:15 +05:00
optional Resolve lint for unused parameter and unnecessary type arguments (#30750) 2024-05-05 08:38:16 +01:00
options
packages Add support for npm bundleDependencies (#30751) 2024-05-05 08:24:01 +01:00
paginator
pprof
private Move database operations of merging a pull request to post receive hook and add a transaction (#30805) 2024-05-12 20:03:10 +02:00
process Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
proxy
proxyprotocol
public
queue refactor: redis queue backend test cleanup 2024-05-21 18:02:33 +02:00
recaptcha
references Fix issue link does not support quotes (#29484) (#29487) 2024-03-06 12:10:45 +08:00
regexplru
repository fix: hook post-receive for sha256 repos 2024-05-15 16:43:16 +02:00
secret [GITEA] Drop sha256-simd in favor of stdlib 2024-02-05 16:09:40 +01:00
session Avoid importing modules/web/middleware in modules/session (#30584) 2024-04-21 16:28:16 +02:00
setting Always load or generate oauth2 jwt secret (#30942) 2024-05-19 15:47:39 +02:00
sitemap
ssh
storage Add minio bucket lookup type option 2024-03-06 09:38:02 +00:00
structs Simplify mirror repository API logic (#30963) 2024-05-19 13:57:52 +02:00
svg
sync
system
templates Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
test [TESTS] add log.Level to test.NewLogChecker 2024-02-05 16:54:44 +01:00
testlogger Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
timeutil Remove the time-since class (#29826) 2024-03-20 08:46:30 +01:00
translation Rename ByteSize to ReadableSize 2024-03-31 20:17:12 +05:00
turnstile
typesniffer Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
updatechecker [PRIVACY] Add a DNS method to fetch new updates 2024-02-05 15:38:36 +01:00
uri
user
util Add codespell support and fix a good number of typos with its help (#3270) 2024-05-09 13:49:37 +00:00
validation added validation fixes 2024-05-14 08:31:34 +02:00
web Avoid importing modules/web/middleware in modules/session (#30584) 2024-04-21 16:28:16 +02:00
webhook [FEAT] sourcehut webhooks 2024-04-05 10:20:40 +02:00