Add an admin API for users' media statistics (#8700)

Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
This commit is contained in:
Dirk Klimpel 2020-11-05 19:59:12 +01:00 committed by GitHub
parent e4676bd877
commit c3119d1536
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 820 additions and 0 deletions

1
changelog.d/8700.feature Normal file
View file

@ -0,0 +1 @@
Add an admin API for local user media statistics. Contributed by @dklimpel.

View file

@ -0,0 +1,83 @@
# Users' media usage statistics
Returns information about all local media usage of users. Gives the
possibility to filter them by time and user.
The API is:
```
GET /_synapse/admin/v1/statistics/users/media
```
To use it, you will need to authenticate by providing an `access_token`
for a server admin: see [README.rst](README.rst).
A response body like the following is returned:
```json
{
"users": [
{
"displayname": "foo_user_0",
"media_count": 2,
"media_length": 134,
"user_id": "@foo_user_0:test"
},
{
"displayname": "foo_user_1",
"media_count": 2,
"media_length": 134,
"user_id": "@foo_user_1:test"
}
],
"next_token": 3,
"total": 10
}
```
To paginate, check for `next_token` and if present, call the endpoint
again with `from` set to the value of `next_token`. This will return a new page.
If the endpoint does not return a `next_token` then there are no more
reports to paginate through.
**Parameters**
The following parameters should be set in the URL:
* `limit`: string representing a positive integer - Is optional but is
used for pagination, denoting the maximum number of items to return
in this call. Defaults to `100`.
* `from`: string representing a positive integer - Is optional but used for pagination,
denoting the offset in the returned results. This should be treated as an opaque value
and not explicitly set to anything other than the return value of `next_token` from a
previous call. Defaults to `0`.
* `order_by` - string - The method in which to sort the returned list of users. Valid values are:
- `user_id` - Users are ordered alphabetically by `user_id`. This is the default.
- `displayname` - Users are ordered alphabetically by `displayname`.
- `media_length` - Users are ordered by the total size of uploaded media in bytes.
Smallest to largest.
- `media_count` - Users are ordered by number of uploaded media. Smallest to largest.
* `from_ts` - string representing a positive integer - Considers only
files created at this timestamp or later. Unix timestamp in ms.
* `until_ts` - string representing a positive integer - Considers only
files created at this timestamp or earlier. Unix timestamp in ms.
* `search_term` - string - Filter users by their user ID localpart **or** displayname.
The search term can be found in any part of the string.
Defaults to no filtering.
* `dir` - string - Direction of order. Either `f` for forwards or `b` for backwards.
Setting this value to `b` will reverse the above sort order. Defaults to `f`.
**Response**
The following fields are returned in the JSON response body:
* `users` - An array of objects, each containing information
about the user and their local media. Objects contain the following fields:
- `displayname` - string - Displayname of this user.
- `media_count` - integer - Number of uploaded media by this user.
- `media_length` - integer - Size of uploaded media in bytes by this user.
- `user_id` - string - Fully-qualified user ID (ex. `@user:server.com`).
* `next_token` - integer - Opaque value used for pagination. See above.
* `total` - integer - Total number of users after filtering.

View file

@ -47,6 +47,7 @@ from synapse.rest.admin.rooms import (
ShutdownRoomRestServlet,
)
from synapse.rest.admin.server_notice_servlet import SendServerNoticeServlet
from synapse.rest.admin.statistics import UserMediaStatisticsRestServlet
from synapse.rest.admin.users import (
AccountValidityRenewServlet,
DeactivateAccountRestServlet,
@ -227,6 +228,7 @@ def register_servlets(hs, http_server):
DeviceRestServlet(hs).register(http_server)
DevicesRestServlet(hs).register(http_server)
DeleteDevicesRestServlet(hs).register(http_server)
UserMediaStatisticsRestServlet(hs).register(http_server)
EventReportDetailRestServlet(hs).register(http_server)
EventReportsRestServlet(hs).register(http_server)
PushersRestServlet(hs).register(http_server)

View file

@ -0,0 +1,122 @@
# -*- coding: utf-8 -*-
# Copyright 2020 Dirk Klimpel
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from typing import TYPE_CHECKING, Tuple
from synapse.api.errors import Codes, SynapseError
from synapse.http.servlet import RestServlet, parse_integer, parse_string
from synapse.http.site import SynapseRequest
from synapse.rest.admin._base import admin_patterns, assert_requester_is_admin
from synapse.storage.databases.main.stats import UserSortOrder
from synapse.types import JsonDict
if TYPE_CHECKING:
from synapse.server import HomeServer
logger = logging.getLogger(__name__)
class UserMediaStatisticsRestServlet(RestServlet):
"""
Get statistics about uploaded media by users.
"""
PATTERNS = admin_patterns("/statistics/users/media$")
def __init__(self, hs: "HomeServer"):
self.hs = hs
self.auth = hs.get_auth()
self.store = hs.get_datastore()
async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
await assert_requester_is_admin(self.auth, request)
order_by = parse_string(
request, "order_by", default=UserSortOrder.USER_ID.value
)
if order_by not in (
UserSortOrder.MEDIA_LENGTH.value,
UserSortOrder.MEDIA_COUNT.value,
UserSortOrder.USER_ID.value,
UserSortOrder.DISPLAYNAME.value,
):
raise SynapseError(
400,
"Unknown value for order_by: %s" % (order_by,),
errcode=Codes.INVALID_PARAM,
)
start = parse_integer(request, "from", default=0)
if start < 0:
raise SynapseError(
400,
"Query parameter from must be a string representing a positive integer.",
errcode=Codes.INVALID_PARAM,
)
limit = parse_integer(request, "limit", default=100)
if limit < 0:
raise SynapseError(
400,
"Query parameter limit must be a string representing a positive integer.",
errcode=Codes.INVALID_PARAM,
)
from_ts = parse_integer(request, "from_ts", default=0)
if from_ts < 0:
raise SynapseError(
400,
"Query parameter from_ts must be a string representing a positive integer.",
errcode=Codes.INVALID_PARAM,
)
until_ts = parse_integer(request, "until_ts")
if until_ts is not None:
if until_ts < 0:
raise SynapseError(
400,
"Query parameter until_ts must be a string representing a positive integer.",
errcode=Codes.INVALID_PARAM,
)
if until_ts <= from_ts:
raise SynapseError(
400,
"Query parameter until_ts must be greater than from_ts.",
errcode=Codes.INVALID_PARAM,
)
search_term = parse_string(request, "search_term")
if search_term == "":
raise SynapseError(
400,
"Query parameter search_term cannot be an empty string.",
errcode=Codes.INVALID_PARAM,
)
direction = parse_string(request, "dir", default="f")
if direction not in ("f", "b"):
raise SynapseError(
400, "Unknown direction: %s" % (direction,), errcode=Codes.INVALID_PARAM
)
users_media, total = await self.store.get_users_media_usage_paginate(
start, limit, from_ts, until_ts, order_by, direction, search_term
)
ret = {"users": users_media, "total": total}
if (start + limit) < total:
ret["next_token"] = start + len(users_media)
return 200, ret

View file

@ -16,15 +16,18 @@
import logging
from collections import Counter
from enum import Enum
from itertools import chain
from typing import Any, Dict, List, Optional, Tuple
from twisted.internet.defer import DeferredLock
from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import StoreError
from synapse.storage.database import DatabasePool
from synapse.storage.databases.main.state_deltas import StateDeltasStore
from synapse.storage.engines import PostgresEngine
from synapse.types import JsonDict
from synapse.util.caches.descriptors import cached
logger = logging.getLogger(__name__)
@ -59,6 +62,23 @@ TYPE_TO_TABLE = {"room": ("room_stats", "room_id"), "user": ("user_stats", "user
TYPE_TO_ORIGIN_TABLE = {"room": ("rooms", "room_id"), "user": ("users", "name")}
class UserSortOrder(Enum):
"""
Enum to define the sorting method used when returning users
with get_users_media_usage_paginate
MEDIA_LENGTH = ordered by size of uploaded media. Smallest to largest.
MEDIA_COUNT = ordered by number of uploaded media. Smallest to largest.
USER_ID = ordered alphabetically by `user_id`.
DISPLAYNAME = ordered alphabetically by `displayname`
"""
MEDIA_LENGTH = "media_length"
MEDIA_COUNT = "media_count"
USER_ID = "user_id"
DISPLAYNAME = "displayname"
class StatsStore(StateDeltasStore):
def __init__(self, database: DatabasePool, db_conn, hs):
super().__init__(database, db_conn, hs)
@ -882,3 +902,110 @@ class StatsStore(StateDeltasStore):
complete_with_stream_id=pos,
absolute_field_overrides={"joined_rooms": joined_rooms},
)
async def get_users_media_usage_paginate(
self,
start: int,
limit: int,
from_ts: Optional[int] = None,
until_ts: Optional[int] = None,
order_by: Optional[UserSortOrder] = UserSortOrder.USER_ID.value,
direction: Optional[str] = "f",
search_term: Optional[str] = None,
) -> Tuple[List[JsonDict], Dict[str, int]]:
"""Function to retrieve a paginated list of users and their uploaded local media
(size and number). This will return a json list of users and the
total number of users matching the filter criteria.
Args:
start: offset to begin the query from
limit: number of rows to retrieve
from_ts: request only media that are created later than this timestamp (ms)
until_ts: request only media that are created earlier than this timestamp (ms)
order_by: the sort order of the returned list
direction: sort ascending or descending
search_term: a string to filter user names by
Returns:
A list of user dicts and an integer representing the total number of
users that exist given this query
"""
def get_users_media_usage_paginate_txn(txn):
filters = []
args = [self.hs.config.server_name]
if search_term:
filters.append("(lmr.user_id LIKE ? OR displayname LIKE ?)")
args.extend(["@%" + search_term + "%:%", "%" + search_term + "%"])
if from_ts:
filters.append("created_ts >= ?")
args.extend([from_ts])
if until_ts:
filters.append("created_ts <= ?")
args.extend([until_ts])
# Set ordering
if UserSortOrder(order_by) == UserSortOrder.MEDIA_LENGTH:
order_by_column = "media_length"
elif UserSortOrder(order_by) == UserSortOrder.MEDIA_COUNT:
order_by_column = "media_count"
elif UserSortOrder(order_by) == UserSortOrder.USER_ID:
order_by_column = "lmr.user_id"
elif UserSortOrder(order_by) == UserSortOrder.DISPLAYNAME:
order_by_column = "displayname"
else:
raise StoreError(
500, "Incorrect value for order_by provided: %s" % order_by
)
if direction == "b":
order = "DESC"
else:
order = "ASC"
where_clause = "WHERE " + " AND ".join(filters) if len(filters) > 0 else ""
sql_base = """
FROM local_media_repository as lmr
LEFT JOIN profiles AS p ON lmr.user_id = '@' || p.user_id || ':' || ?
{}
GROUP BY lmr.user_id, displayname
""".format(
where_clause
)
# SQLite does not support SELECT COUNT(*) OVER()
sql = """
SELECT COUNT(*) FROM (
SELECT lmr.user_id
{sql_base}
) AS count_user_ids
""".format(
sql_base=sql_base,
)
txn.execute(sql, args)
count = txn.fetchone()[0]
sql = """
SELECT
lmr.user_id,
displayname,
COUNT(lmr.user_id) as media_count,
SUM(media_length) as media_length
{sql_base}
ORDER BY {order_by_column} {order}
LIMIT ? OFFSET ?
""".format(
sql_base=sql_base, order_by_column=order_by_column, order=order,
)
args += [limit, start]
txn.execute(sql, args)
users = self.db_pool.cursor_to_dict(txn)
return users, count
return await self.db_pool.runInteraction(
"get_users_media_usage_paginate_txn", get_users_media_usage_paginate_txn
)

View file

@ -0,0 +1,485 @@
# -*- coding: utf-8 -*-
# Copyright 2020 Dirk Klimpel
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
from binascii import unhexlify
from typing import Any, Dict, List, Optional
import synapse.rest.admin
from synapse.api.errors import Codes
from synapse.rest.client.v1 import login
from tests import unittest
class UserMediaStatisticsTestCase(unittest.HomeserverTestCase):
servlets = [
synapse.rest.admin.register_servlets,
login.register_servlets,
]
def prepare(self, reactor, clock, hs):
self.store = hs.get_datastore()
self.media_repo = hs.get_media_repository_resource()
self.admin_user = self.register_user("admin", "pass", admin=True)
self.admin_user_tok = self.login("admin", "pass")
self.other_user = self.register_user("user", "pass")
self.other_user_tok = self.login("user", "pass")
self.url = "/_synapse/admin/v1/statistics/users/media"
def test_no_auth(self):
"""
Try to list users without authentication.
"""
request, channel = self.make_request("GET", self.url, b"{}")
self.render(request)
self.assertEqual(401, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.MISSING_TOKEN, channel.json_body["errcode"])
def test_requester_is_no_admin(self):
"""
If the user is not a server admin, an error 403 is returned.
"""
request, channel = self.make_request(
"GET", self.url, json.dumps({}), access_token=self.other_user_tok,
)
self.render(request)
self.assertEqual(403, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.FORBIDDEN, channel.json_body["errcode"])
def test_invalid_parameter(self):
"""
If parameters are invalid, an error is returned.
"""
# unkown order_by
request, channel = self.make_request(
"GET", self.url + "?order_by=bar", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# negative from
request, channel = self.make_request(
"GET", self.url + "?from=-5", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# negative limit
request, channel = self.make_request(
"GET", self.url + "?limit=-5", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# negative from_ts
request, channel = self.make_request(
"GET", self.url + "?from_ts=-1234", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# negative until_ts
request, channel = self.make_request(
"GET", self.url + "?until_ts=-1234", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# until_ts smaller from_ts
request, channel = self.make_request(
"GET",
self.url + "?from_ts=10&until_ts=5",
access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# empty search term
request, channel = self.make_request(
"GET", self.url + "?search_term=", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
# invalid search order
request, channel = self.make_request(
"GET", self.url + "?dir=bar", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(400, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(Codes.INVALID_PARAM, channel.json_body["errcode"])
def test_limit(self):
"""
Testing list of media with limit
"""
self._create_users_with_media(10, 2)
request, channel = self.make_request(
"GET", self.url + "?limit=5", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 10)
self.assertEqual(len(channel.json_body["users"]), 5)
self.assertEqual(channel.json_body["next_token"], 5)
self._check_fields(channel.json_body["users"])
def test_from(self):
"""
Testing list of media with a defined starting point (from)
"""
self._create_users_with_media(20, 2)
request, channel = self.make_request(
"GET", self.url + "?from=5", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 20)
self.assertEqual(len(channel.json_body["users"]), 15)
self.assertNotIn("next_token", channel.json_body)
self._check_fields(channel.json_body["users"])
def test_limit_and_from(self):
"""
Testing list of media with a defined starting point and limit
"""
self._create_users_with_media(20, 2)
request, channel = self.make_request(
"GET", self.url + "?from=5&limit=10", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 20)
self.assertEqual(channel.json_body["next_token"], 15)
self.assertEqual(len(channel.json_body["users"]), 10)
self._check_fields(channel.json_body["users"])
def test_next_token(self):
"""
Testing that `next_token` appears at the right place
"""
number_users = 20
self._create_users_with_media(number_users, 3)
# `next_token` does not appear
# Number of results is the number of entries
request, channel = self.make_request(
"GET", self.url + "?limit=20", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], number_users)
self.assertEqual(len(channel.json_body["users"]), number_users)
self.assertNotIn("next_token", channel.json_body)
# `next_token` does not appear
# Number of max results is larger than the number of entries
request, channel = self.make_request(
"GET", self.url + "?limit=21", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], number_users)
self.assertEqual(len(channel.json_body["users"]), number_users)
self.assertNotIn("next_token", channel.json_body)
# `next_token` does appear
# Number of max results is smaller than the number of entries
request, channel = self.make_request(
"GET", self.url + "?limit=19", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], number_users)
self.assertEqual(len(channel.json_body["users"]), 19)
self.assertEqual(channel.json_body["next_token"], 19)
# Set `from` to value of `next_token` for request remaining entries
# Check `next_token` does not appear
request, channel = self.make_request(
"GET", self.url + "?from=19", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], number_users)
self.assertEqual(len(channel.json_body["users"]), 1)
self.assertNotIn("next_token", channel.json_body)
def test_no_media(self):
"""
Tests that a normal lookup for statistics is successfully
if users have no media created
"""
request, channel = self.make_request(
"GET", self.url, access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, channel.code, msg=channel.json_body)
self.assertEqual(0, channel.json_body["total"])
self.assertEqual(0, len(channel.json_body["users"]))
def test_order_by(self):
"""
Testing order list with parameter `order_by`
"""
# create users
self.register_user("user_a", "pass", displayname="UserZ")
userA_tok = self.login("user_a", "pass")
self._create_media(userA_tok, 1)
self.register_user("user_b", "pass", displayname="UserY")
userB_tok = self.login("user_b", "pass")
self._create_media(userB_tok, 3)
self.register_user("user_c", "pass", displayname="UserX")
userC_tok = self.login("user_c", "pass")
self._create_media(userC_tok, 2)
# order by user_id
self._order_test("user_id", ["@user_a:test", "@user_b:test", "@user_c:test"])
self._order_test(
"user_id", ["@user_a:test", "@user_b:test", "@user_c:test"], "f",
)
self._order_test(
"user_id", ["@user_c:test", "@user_b:test", "@user_a:test"], "b",
)
# order by displayname
self._order_test(
"displayname", ["@user_c:test", "@user_b:test", "@user_a:test"]
)
self._order_test(
"displayname", ["@user_c:test", "@user_b:test", "@user_a:test"], "f",
)
self._order_test(
"displayname", ["@user_a:test", "@user_b:test", "@user_c:test"], "b",
)
# order by media_length
self._order_test(
"media_length", ["@user_a:test", "@user_c:test", "@user_b:test"],
)
self._order_test(
"media_length", ["@user_a:test", "@user_c:test", "@user_b:test"], "f",
)
self._order_test(
"media_length", ["@user_b:test", "@user_c:test", "@user_a:test"], "b",
)
# order by media_count
self._order_test(
"media_count", ["@user_a:test", "@user_c:test", "@user_b:test"],
)
self._order_test(
"media_count", ["@user_a:test", "@user_c:test", "@user_b:test"], "f",
)
self._order_test(
"media_count", ["@user_b:test", "@user_c:test", "@user_a:test"], "b",
)
def test_from_until_ts(self):
"""
Testing filter by time with parameters `from_ts` and `until_ts`
"""
# create media earlier than `ts1` to ensure that `from_ts` is working
self._create_media(self.other_user_tok, 3)
self.pump(1)
ts1 = self.clock.time_msec()
# list all media when filter is not set
request, channel = self.make_request(
"GET", self.url, access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["users"][0]["media_count"], 3)
# filter media starting at `ts1` after creating first media
# result is 0
request, channel = self.make_request(
"GET", self.url + "?from_ts=%s" % (ts1,), access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 0)
self._create_media(self.other_user_tok, 3)
self.pump(1)
ts2 = self.clock.time_msec()
# create media after `ts2` to ensure that `until_ts` is working
self._create_media(self.other_user_tok, 3)
# filter media between `ts1` and `ts2`
request, channel = self.make_request(
"GET",
self.url + "?from_ts=%s&until_ts=%s" % (ts1, ts2),
access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["users"][0]["media_count"], 3)
# filter media until `ts2` and earlier
request, channel = self.make_request(
"GET", self.url + "?until_ts=%s" % (ts2,), access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["users"][0]["media_count"], 6)
def test_search_term(self):
self._create_users_with_media(20, 1)
# check without filter get all users
request, channel = self.make_request(
"GET", self.url, access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 20)
# filter user 1 and 10-19 by `user_id`
request, channel = self.make_request(
"GET",
self.url + "?search_term=foo_user_1",
access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 11)
# filter on this user in `displayname`
request, channel = self.make_request(
"GET",
self.url + "?search_term=bar_user_10",
access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["users"][0]["displayname"], "bar_user_10")
self.assertEqual(channel.json_body["total"], 1)
# filter and get empty result
request, channel = self.make_request(
"GET", self.url + "?search_term=foobar", access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, int(channel.result["code"]), msg=channel.result["body"])
self.assertEqual(channel.json_body["total"], 0)
def _create_users_with_media(self, number_users: int, media_per_user: int):
"""
Create a number of users with a number of media
Args:
number_users: Number of users to be created
media_per_user: Number of media to be created for each user
"""
for i in range(number_users):
self.register_user("foo_user_%s" % i, "pass", displayname="bar_user_%s" % i)
user_tok = self.login("foo_user_%s" % i, "pass")
self._create_media(user_tok, media_per_user)
def _create_media(self, user_token: str, number_media: int):
"""
Create a number of media for a specific user
Args:
user_token: Access token of the user
number_media: Number of media to be created for the user
"""
upload_resource = self.media_repo.children[b"upload"]
for i in range(number_media):
# file size is 67 Byte
image_data = unhexlify(
b"89504e470d0a1a0a0000000d4948445200000001000000010806"
b"0000001f15c4890000000a49444154789c63000100000500010d"
b"0a2db40000000049454e44ae426082"
)
# Upload some media into the room
self.helper.upload_media(
upload_resource, image_data, tok=user_token, expect_code=200
)
def _check_fields(self, content: List[Dict[str, Any]]):
"""Checks that all attributes are present in content
Args:
content: List that is checked for content
"""
for c in content:
self.assertIn("user_id", c)
self.assertIn("displayname", c)
self.assertIn("media_count", c)
self.assertIn("media_length", c)
def _order_test(
self, order_type: str, expected_user_list: List[str], dir: Optional[str] = None
):
"""Request the list of users in a certain order. Assert that order is what
we expect
Args:
order_type: The type of ordering to give the server
expected_user_list: The list of user_ids in the order we expect to get
back from the server
dir: The direction of ordering to give the server
"""
url = self.url + "?order_by=%s" % (order_type,)
if dir is not None and dir in ("b", "f"):
url += "&dir=%s" % (dir,)
request, channel = self.make_request(
"GET", url.encode("ascii"), access_token=self.admin_user_tok,
)
self.render(request)
self.assertEqual(200, channel.code, msg=channel.json_body)
self.assertEqual(channel.json_body["total"], len(expected_user_list))
returned_order = [row["user_id"] for row in channel.json_body["users"]]
self.assertListEqual(expected_user_list, returned_order)
self._check_fields(channel.json_body["users"])