Compare commits

..

38 Commits

Author SHA1 Message Date
diced 88fdb2fcc1 fix: unecessary stuff 2023-05-29 17:55:39 -07:00
diced e92d78f671 fix: no thumbnailId 2023-05-28 22:20:15 -07:00
dicedtomato bcd2897c4e Merge branch 'trunk' into feature/vid-thumb 2023-05-28 21:38:49 -07:00
diced 24dacb478d feat: thumbnails final 2023-05-28 21:38:27 -07:00
diced 4893f4a09e feat: thumbnails workers 2023-05-23 22:46:46 -07:00
Jayvin Hernandez f40d65a9f7 feat: view other user files (#408)
* feat: Add the capability of viewing another user's images as admin.

* fix: add columns, oops...

* fix: Gotta check if the user's legit before letting them see

* fix: made administrators non-viewable

* Please don't reference yourself

* fix: superAdmin > admin

---------

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-23 22:34:21 -07:00
Jayvin Hernandez a2c085719a feat: new opt bug.yml (#411)
Hopefully this is a better explanation for the version selector.

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-22 15:36:53 -07:00
Jayvin Hernandez 60d7b22dca fix: ability to delete other users images (#407)
* fix: Worst, but minimally working, fix so other users do not delete each other's files.

* fix: include previous fix for PATCH

---------

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-22 15:36:19 -07:00
Jayvin Hernandez d111b0811f fix: add a debug log (#406) 2023-05-22 15:32:26 -07:00
NebulaBC b46e7b8ba2 Fix branding oversight caused by 0a34b0c (#405) 2023-05-16 14:46:48 -07:00
diced 39a8d52353 feat: built-in robots.txt (#402) 2023-05-13 16:25:27 -07:00
diced ec09458ad3 feat: allow full paths (#393) 2023-05-13 00:12:29 -07:00
Jayvin Hernandez a7ad58b196 fix: embed links #390 (#403)
* fix: Use raw route for embedding images.

* fix: there was already a raw link passed 😔
2023-05-12 23:14:59 -07:00
Jayvin Hernandez 1ddd351242 fix: things (#401)
* fix: use relative path for volumes

* fix: null check oauth config

* fix: attempt refresh if error is on /dashboard

---------

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-10 22:39:08 -07:00
Derock 24b06c76fb feat: server-side sorting (#366)
Co-authored-by: Jayvin Hernandez <gogojayvin923@gmail.com>
Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-10 22:32:13 -07:00
Dane 0a34b0cc21 feat: bypass local login (#373)
* adding option to bypass local login

* oops

* fix: add descriptive title

---------

Co-authored-by: Jayvin Hernandez <gogojayvin923@gmail.com>
Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-05-06 11:04:52 -07:00
diced ce26a414ac fix: update reference (#392) 2023-05-06 10:41:46 -07:00
diced f71aab2cde fix: body size limit for pfp (#389) 2023-05-06 10:37:38 -07:00
diced 5f76e9d383 refactor: copy errors removed (#344) 2023-05-06 10:36:08 -07:00
diced 4a46f15833 fix: ext finding (#384) 2023-04-30 15:41:27 -07:00
diced d6ce64ae21 fix: go back one page (#371) 2023-04-30 15:31:19 -07:00
Jayvin Hernandez 7cbf828f3b fix: properly type the relation field for OAuth (#370)
Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-04-28 20:51:34 -07:00
Jayvin Hernandez 3ff215366a fix: a lot of things (#386)
* @diced forgor

* check if proper id

* fix(?): await datasource's get

* fix (diced/zipline#350): Return size 0 for not found
2023-04-28 20:32:51 -07:00
IThundxr d238e24f62 feat: Add check to prevent favorited files from being deleted (#369)
* additions

* add proper lock system

* migreation

* Revert "migreation"

This reverts commit 4058146c28.

* remove that

* get rid of bad code

---------

Co-authored-by: Jayvin Hernandez <gogojayvin923@gmail.com>
2023-04-12 16:59:30 -07:00
Jayvin Hernandez fd2746c2d0 fix: clearing (#367)
+ added convenient clear-temp script, no flags yet
2023-04-11 18:37:38 -07:00
Jayvin Hernandez 61b2eff6a4 fix: #364
* some fixes for diced/zipline-docs#50

* appropriately changed items

* tight fit for a selector
2023-04-06 20:02:07 -07:00
Jayvin Hernandez 89a28bf50b fix: query-size (#363)
* feat: add flags for querying & delete option

* return 0 for no file

* include size
2023-04-05 20:25:19 -07:00
Jayvin Hernandez 5ded128263 fix: user uuid (#355)
* fix: user uuid is used instead of user id for its uniqueness

* fix: use cuid instead & exclude from parser

* fix: apply new foreign key constraints to existing data

* fix: migration partly done

* not-fix: General form of migration achieved, still broken

* fix: migrate and use db's uuid function for existing users

* fix: Proper not nulling!

* fix: #354

* fix: migration & use uuid instead

---------

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
Co-authored-by: diced <pranaco2@gmail.com>
2023-04-04 20:07:41 -07:00
dicedtomato eedeb89c7d feat: offloaded chunked uploads (#356)
* feat: offloaded chunked uploads

* fix: use temp_directory instead of tmpdir()

* feat: CHUNKS_ENABLED config
2023-04-03 22:42:27 -07:00
Jayvin Hernandez bf40fa9cd2 feat: many things (#351)
* remove source from final image

* move check state to ClearStorage

* use inspect for fancy colors

* newlines are now possible! yay!

* Catch user's leave if uploading

* feat?: Temp directory can be specified by the user.
Default is /tmp/zipline (or os equivalent)

* fix: ignore onDash config, use only ?compress query

---------

Co-authored-by: dicedtomato <35403473+diced@users.noreply.github.com>
2023-03-31 22:25:00 -07:00
diced bc58c1b56e fix: milestone again again again again again again 2023-03-31 22:12:13 -07:00
diced c57a6e1700 fix: milestone again again again again again 2023-03-31 22:07:17 -07:00
diced 8649a489d8 fix: milestone again again again again 2023-03-31 22:05:59 -07:00
diced 40f29907c7 fix: milestone again again again 2023-03-31 21:55:26 -07:00
diced 34005ece43 fix: milestone again again 2023-03-31 21:53:30 -07:00
diced 8e6fc1e8a3 fix: milestone again 2023-03-31 21:49:50 -07:00
diced 065f44b145 fix: milestone 2023-03-31 21:41:19 -07:00
diced e5a07f568d fix: update milestone action 2023-03-27 16:48:56 -07:00
77 changed files with 1578 additions and 459 deletions
+21 -17
View File
@@ -1,46 +1,50 @@
# every field in here is optional except, CORE_SECRET and CORE_DATABASE_URL.
# if CORE_SECRET is still "changethis" then zipline will exit and tell you to change it.
# if using s3/supabase make sure to comment out the other datasources
# if using s3/supabase make sure to uncomment or comment out the correct lines needed.
CORE_HTTPS=true
CORE_RETURN_HTTPS=true
CORE_SECRET="changethis"
CORE_HOST=0.0.0.0
CORE_PORT=3000
CORE_DATABASE_URL="postgres://postgres:postgres@localhost/zip10"
CORE_LOGGER=false
CORE_STATS_INTERVAL=1800
CORE_INVITES_INTERVAL=1800
CORE_THUMBNAILS_INTERVAL=600
# default
DATASOURCE_TYPE=local
DATASOURCE_LOCAL_DIRECTORY=./uploads
# or you can choose to use s3
DATASOURCE_TYPE=s3
DATASOURCE_S3_ACCESS_KEY_ID=key
DATASOURCE_S3_SECRET_ACCESS_KEY=secret
DATASOURCE_S3_BUCKET=bucket
DATASOURCE_S3_ENDPOINT=s3.amazonaws.com
DATASOURCE_S3_REGION=us-west-2
DATASOURCE_S3_FORCE_S3_PATH=false
DATASOURCE_S3_USE_SSL=false
# DATASOURCE_TYPE=s3
# DATASOURCE_S3_ACCESS_KEY_ID=key
# DATASOURCE_S3_SECRET_ACCESS_KEY=secret
# DATASOURCE_S3_BUCKET=bucket
# DATASOURCE_S3_ENDPOINT=s3.amazonaws.com
# DATASOURCE_S3_REGION=us-west-2
# DATASOURCE_S3_FORCE_S3_PATH=false
# DATASOURCE_S3_USE_SSL=false
# or supabase
DATASOURCE_TYPE=supabase
DATASOURCE_SUPABASE_KEY=xxx
# DATASOURCE_TYPE=supabase
# DATASOURCE_SUPABASE_KEY=xxx
# remember: no leading slash
DATASOURCE_SUPABASE_URL=https://something.supabase.co
DATASOURCE_SUPABASE_BUCKET=zipline
# DATASOURCE_SUPABASE_URL=https://something.supabase.co
# DATASOURCE_SUPABASE_BUCKET=zipline
UPLOADER_DEFAULT_FORMAT=RANDOM
UPLOADER_ROUTE=/u
UPLOADER_LENGTH=6
UPLOADER_ADMIN_LIMIT=104900000
UPLOADER_USER_LIMIT=104900000
UPLOADER_DISABLED_EXTENSIONS=someext
UPLOADER_DISABLED_EXTENSIONS=someext,anotherext
URLS_ROUTE=/go
URLS_LENGTH=6
RATELIMIT_USER = 5
RATELIMIT_ADMIN = 3
RATELIMIT_USER=5
RATELIMIT_ADMIN=3
# for more variables checkout the docs
+2 -2
View File
@@ -15,10 +15,10 @@ body:
id: version
attributes:
label: Version
description: What version of Zipline are you using?
description: What version (or docker image) of Zipline are you using?
options:
- latest (ghcr.io/diced/zipline or ghcr.io/diced/zipline:latest)
- upstream (ghcr.io/diced/zipline:trunk)
- latest (ghcr.io/diced/zipline:latest)
- other (provide version in additional info)
validations:
required: true
+1 -1
View File
@@ -1,7 +1,7 @@
blank_issues_enabled: false
contact_links:
- name: Feature Request
url: https://github.com/diced/zipline/discussions/new?category=ideas&title=Your%20breif%20description%20here&labels=feature
url: https://github.com/diced/zipline/discussions/new?category=ideas&title=Your%20brief%20description%20here&labels=feature
about: Ask for a new feature
- name: Zipline Discord
url: https://discord.gg/EAhCRfGxCF
+12 -5
View File
@@ -1,24 +1,31 @@
name: 'Issue/PR Milestones'
on:
pull_request:
pull_request_target:
types: [opened, reopened]
issues:
types: [opened, reopened]
permissions:
issues: write
checks: write
contents: read
pull-requests: write
jobs:
set:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/github-script@v3
- uses: actions/github-script@v6
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const milestone = 2
github.issues.update({
const milestone = 3
github.rest.issues.update({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
milestone
})
})
+1 -1
View File
@@ -14,7 +14,7 @@ Create an issue on GitHub, please include the following (if one of them is not a
Create an discussion on GitHub, please include the following:
- Breif explanation of the feature in the title (very breif please)
- Brief explanation of the feature in the title (very brief please)
- How it would work (detailed, but optional)
## Pull Requests (contributions to the codebase)
+2 -1
View File
@@ -69,7 +69,8 @@ COPY --from=builder /zipline/node_modules/@prisma/client ./node_modules/@prisma/
# Copy Startup Script
COPY docker-entrypoint.sh /zipline
# Make Startup Script Executable
RUN chmod a+x /zipline/docker-entrypoint.sh
RUN chmod a+x /zipline/docker-entrypoint.sh && rm -rf /zipline/src
# Set the entrypoint to the startup script
ENTRYPOINT ["tini", "--", "/zipline/docker-entrypoint.sh"]
+2 -2
View File
@@ -23,8 +23,8 @@ services:
env_file:
- .env.local
volumes:
- '$PWD/uploads:/zipline/uploads'
- '$PWD/public:/zipline/public'
- './uploads:/zipline/uploads'
- './public:/zipline/public'
depends_on:
- 'postgres'
+1 -1
View File
@@ -29,7 +29,7 @@ services:
- CORE_LOGGER=true
volumes:
- './uploads:/zipline/uploads'
- '$PWD/public:/zipline/public'
- './public:/zipline/public'
depends_on:
- 'postgres'
+3 -1
View File
@@ -24,7 +24,8 @@
"scripts:list-users": "node --enable-source-maps dist/scripts/list-users",
"scripts:set-user": "node --enable-source-maps dist/scripts/set-user",
"scripts:clear-zero-byte": "node --enable-source-maps dist/scripts/clear-zero-byte",
"scripts:query-size": "node --enable-source-maps dist/scripts/query-size"
"scripts:query-size": "node --enable-source-maps dist/scripts/query-size",
"scripts:clear-temp": "node --enable-source-maps dist/scripts/clear-temp"
},
"dependencies": {
"@emotion/react": "^11.10.6",
@@ -53,6 +54,7 @@
"fastify": "^4.15.0",
"fastify-plugin": "^4.5.0",
"fflate": "^0.7.4",
"ffmpeg-static": "^5.1.0",
"find-my-way": "^7.6.0",
"katex": "^0.16.4",
"mantine-datatable": "^2.2.6",
@@ -0,0 +1,18 @@
-- CreateEnum
CREATE TYPE "ProcessingStatus" AS ENUM ('PENDING', 'PROCESSING', 'COMPLETE');
-- CreateTable
CREATE TABLE "IncompleteFile" (
"id" SERIAL NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"status" "ProcessingStatus" NOT NULL,
"chunks" INTEGER NOT NULL,
"chunksComplete" INTEGER NOT NULL,
"userId" INTEGER NOT NULL,
"data" JSONB NOT NULL,
CONSTRAINT "IncompleteFile_pkey" PRIMARY KEY ("id")
);
-- AddForeignKey
ALTER TABLE "IncompleteFile" ADD CONSTRAINT "IncompleteFile_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
@@ -0,0 +1,53 @@
/*
Warnings:
- A unique constraint covering the columns `[uuid]` on the table `User` will be added. If there are existing duplicate values, this will fail.
*/
-- PRISMA GENERATED BELOW
-- -- DropForeignKey
-- ALTER TABLE "OAuth" DROP CONSTRAINT "OAuth_userId_fkey";
--
-- -- AlterTable
-- ALTER TABLE "OAuth" ALTER COLUMN "userId" SET DATA TYPE TEXT;
--
-- -- AlterTable
-- ALTER TABLE "User" ADD COLUMN "uuid" UUID NOT NULL DEFAULT gen_random_uuid();
--
-- -- CreateIndex
-- CREATE UNIQUE INDEX "User_uuid_key" ON "User"("uuid");
--
-- -- AddForeignKey
-- ALTER TABLE "OAuth" ADD CONSTRAINT "OAuth_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("uuid") ON DELETE CASCADE ON UPDATE CASCADE;
-- User made changes below
-- Rename old foreign key
ALTER TABLE "OAuth" RENAME CONSTRAINT "OAuth_userId_fkey" TO "OAuth_userId_old_fkey";
-- Rename old column
ALTER TABLE "OAuth" RENAME COLUMN "userId" TO "userId_old";
-- Add new column
ALTER TABLE "OAuth" ADD COLUMN "userId" UUID;
-- Add user uuid
ALTER TABLE "User" ADD COLUMN "uuid" UUID NOT NULL DEFAULT gen_random_uuid();
-- Update table "OAuth" with uuid
UPDATE "OAuth" SET "userId" = "User"."uuid" FROM "User" WHERE "OAuth"."userId_old" = "User"."id";
-- Alter table "OAuth" to make "userId" required
ALTER TABLE "OAuth" ALTER COLUMN "userId" SET NOT NULL;
-- Create index
CREATE UNIQUE INDEX "User_uuid_key" ON "User"("uuid");
-- Add new foreign key
ALTER TABLE "OAuth" ADD CONSTRAINT "OAuth_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("uuid") ON DELETE CASCADE ON UPDATE CASCADE;
-- Drop old foreign key
ALTER TABLE "OAuth" DROP CONSTRAINT "OAuth_userId_old_fkey";
-- Drop old column
ALTER TABLE "OAuth" DROP COLUMN "userId_old";
@@ -0,0 +1,16 @@
-- CreateTable
CREATE TABLE "Thumbnail" (
"id" SERIAL NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"name" TEXT NOT NULL,
"fileId" INTEGER NOT NULL,
CONSTRAINT "Thumbnail_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX "Thumbnail_fileId_key" ON "Thumbnail"("fileId");
-- AddForeignKey
ALTER TABLE "Thumbnail" ADD CONSTRAINT "Thumbnail_fileId_fkey" FOREIGN KEY ("fileId") REFERENCES "File"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+52 -19
View File
@@ -8,23 +8,25 @@ generator client {
}
model User {
id Int @id @default(autoincrement())
username String
password String?
avatar String?
token String
administrator Boolean @default(false)
superAdmin Boolean @default(false)
systemTheme String @default("system")
embed Json @default("{}")
ratelimit DateTime?
totpSecret String?
domains String[]
oauth OAuth[]
files File[]
urls Url[]
Invite Invite[]
Folder Folder[]
id Int @id @default(autoincrement())
uuid String @unique @default(dbgenerated("gen_random_uuid()")) @db.Uuid
username String
password String?
avatar String?
token String
administrator Boolean @default(false)
superAdmin Boolean @default(false)
systemTheme String @default("system")
embed Json @default("{}")
ratelimit DateTime?
totpSecret String?
domains String[]
oauth OAuth[]
files File[]
urls Url[]
Invite Invite[]
Folder Folder[]
IncompleteFile IncompleteFile[]
}
model Folder {
@@ -60,6 +62,17 @@ model File {
folder Folder? @relation(fields: [folderId], references: [id], onDelete: SetNull)
folderId Int?
thumbnail Thumbnail?
}
model Thumbnail {
id Int @id @default(autoincrement())
createdAt DateTime @default(now())
name String
fileId Int @unique
file File @relation(fields: [fileId], references: [id], onDelete: Cascade)
}
model InvisibleFile {
@@ -111,8 +124,8 @@ model Invite {
model OAuth {
id Int @id @default(autoincrement())
provider OauthProviders
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
userId Int
user User @relation(fields: [userId], references: [uuid], onDelete: Cascade)
userId String @db.Uuid
username String
oauthId String?
token String
@@ -126,3 +139,23 @@ enum OauthProviders {
GITHUB
GOOGLE
}
model IncompleteFile {
id Int @id @default(autoincrement())
createdAt DateTime @default(now())
status ProcessingStatus
chunks Int
chunksComplete Int
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
userId Int
data Json
}
enum ProcessingStatus {
PENDING
PROCESSING
COMPLETE
}
+9 -13
View File
@@ -49,6 +49,7 @@ export default function FileModal({
reducedActions = false,
exifEnabled,
compress,
otherUser = false,
}: {
open: boolean;
setOpen: (open: boolean) => void;
@@ -58,6 +59,7 @@ export default function FileModal({
reducedActions?: boolean;
exifEnabled?: boolean;
compress: boolean;
otherUser: boolean;
}) {
const deleteFile = useFileDelete();
const favoriteFile = useFileFavorite();
@@ -95,18 +97,12 @@ export default function FileModal({
const handleCopy = () => {
clipboard.copy(`${window.location.protocol}//${window.location.host}${file.url}`);
setOpen(false);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
};
const handleFavorite = async () => {
@@ -282,7 +278,7 @@ export default function FileModal({
</ActionIcon>
</Tooltip>
)}
{reducedActions ? null : inFolder && !folders.isLoading ? (
{reducedActions || otherUser ? null : inFolder && !folders.isLoading ? (
<Tooltip
label={`Remove from folder "${folders.data.find((f) => f.id === file.folderId)?.name ?? ''}"`}
>
+17 -3
View File
@@ -32,9 +32,10 @@ export default function File({
image,
disableMediaPreview,
exifEnabled,
refreshImages,
refreshImages = undefined,
reducedActions = false,
onDash,
otherUser = false,
}) {
const [open, setOpen] = useState(false);
const deleteFile = useFileDelete();
@@ -44,7 +45,7 @@ export default function File({
const folders = useFolders();
const refresh = () => {
refreshImages();
if (!otherUser) refreshImages();
folders.refetch();
};
@@ -59,9 +60,22 @@ export default function File({
reducedActions={reducedActions}
exifEnabled={exifEnabled}
compress={onDash}
otherUser={otherUser}
/>
<Card sx={{ maxWidth: '100%', height: '100%' }} shadow='md' onClick={() => setOpen(true)}>
<Card
sx={{
maxWidth: '100%',
height: '100%',
'&:hover': {
filter: 'brightness(0.75)',
},
transition: 'filter 0.2s ease-in-out',
cursor: 'pointer',
}}
shadow='md'
onClick={() => setOpen(true)}
>
<Card.Section>
<LoadingOverlay visible={loading} />
<Type
+5 -14
View File
@@ -4,10 +4,8 @@ import {
Box,
Burger,
Button,
Group,
Header,
Image,
Input,
MediaQuery,
Menu,
Navbar,
@@ -220,21 +218,14 @@ export default function Layout({ children, props }) {
labels: { confirm: 'Copy', cancel: 'Cancel' },
onConfirm: async () => {
clipboard.copy(token);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: (
<Text size='sm'>
Zipline is unable to copy to clipboard due to security reasons. However, you can still copy
the token manually.
<br />
<Group position='left' spacing='sm'>
<Text>Your token is:</Text>
<Input size='sm' onFocus={(e) => e.target.select()} type='text' value={token} />
</Group>
</Text>
),
title: 'Unable to copy token',
message:
"Zipline couldn't copy to your clipboard. Please copy the token manually from the settings page.",
color: 'red',
icon: <IconClipboardCopy size='1rem' />,
});
else
showNotification({
+27 -1
View File
@@ -53,6 +53,31 @@ function Placeholder({ text, Icon, ...props }) {
);
}
function VideoThumbnailPlaceholder({ file, mediaPreview, ...props }) {
if (!file.thumbnail || !mediaPreview)
return <Placeholder Icon={IconPlayerPlay} text={`Click to view video (${file.name})`} {...props} />;
return (
<Box>
<Image
src={file.thumbnail}
sx={{
position: 'absolute',
width: '100%',
height: '100%',
objectFit: 'cover',
}}
/>
<Center sx={{ position: 'absolute', width: '100%', height: '100%' }}>
<IconPlayerPlay size={48} />
</Center>
</Box>
// </Placeholder>
);
}
export default function Type({ file, popup = false, disableMediaPreview, ...props }) {
const type =
(file.type ?? file.mimetype) === ''
@@ -159,7 +184,8 @@ export default function Type({ file, popup = false, disableMediaPreview, ...prop
)
) : media ? (
{
video: <Placeholder Icon={IconPlayerPlay} text={`Click to view video (${file.name})`} {...props} />,
// video: <Placeholder Icon={IconPlayerPlay} text={`Click to view video (${file.name})`} {...props} />,
video: <VideoThumbnailPlaceholder file={file} mediaPreview={!disableMediaPreview} />,
image: (
<Image
placeholder={<PlaceholderContent Icon={IconPhotoCancel} text={'Image failed to load...'} />}
+25 -38
View File
@@ -12,7 +12,7 @@ import {
import FileModal from 'components/File/FileModal';
import MutedText from 'components/MutedText';
import useFetch from 'lib/hooks/useFetch';
import { usePaginatedFiles, useRecent } from 'lib/queries/files';
import { PaginatedFilesOptions, usePaginatedFiles, useRecent } from 'lib/queries/files';
import { useStats } from 'lib/queries/stats';
import { userSelector } from 'lib/recoil/user';
import { bytesToHuman } from 'lib/utils/bytes';
@@ -45,32 +45,24 @@ export default function Dashboard({ disableMediaPreview, exifEnabled, compress }
})();
}, [page]);
const files = usePaginatedFiles(page, 'none');
// sorting
const [sortStatus, setSortStatus] = useState<DataTableSortStatus>({
columnAccessor: 'date',
columnAccessor: 'createdAt',
direction: 'asc',
});
const [records, setRecords] = useState(files.data);
useEffect(() => {
setRecords(files.data);
}, [files.data]);
const files = usePaginatedFiles(page, {
filter: 'none',
useEffect(() => {
if (!records || records.length === 0) return;
const sortedRecords = [...records].sort((a, b) => {
if (sortStatus.direction === 'asc') {
return a[sortStatus.columnAccessor] > b[sortStatus.columnAccessor] ? 1 : -1;
}
return a[sortStatus.columnAccessor] < b[sortStatus.columnAccessor] ? 1 : -1;
});
setRecords(sortedRecords);
}, [sortStatus]);
// only query for correct results if there is more than one page
// otherwise, querying has no effect
...(numFiles > 1
? {
sortBy: sortStatus.columnAccessor as PaginatedFilesOptions['sortBy'],
order: sortStatus.direction,
}
: {}),
});
// file modal on click
const [open, setOpen] = useState(false);
@@ -106,22 +98,16 @@ export default function Dashboard({ disableMediaPreview, exifEnabled, compress }
const copyFile = async (file) => {
clipboard.copy(`${window.location.protocol}//${window.location.host}${file.url}`);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: (
<a
href={`${window.location.protocol}//${window.location.host}${file.url}`}
>{`${window.location.protocol}//${window.location.host}${file.url}`}</a>
),
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: (
<a
href={`${window.location.protocol}//${window.location.host}${file.url}`}
>{`${window.location.protocol}//${window.location.host}${file.url}`}</a>
),
icon: <IconClipboardCopy size='1rem' />,
});
};
const viewFile = async (file) => {
@@ -140,6 +126,7 @@ export default function Dashboard({ disableMediaPreview, exifEnabled, compress }
reducedActions={false}
exifEnabled={exifEnabled}
compress={compress}
otherUser={false}
/>
)}
@@ -209,7 +196,7 @@ export default function Dashboard({ disableMediaPreview, exifEnabled, compress }
),
},
]}
records={records ?? []}
records={files.data ?? []}
fetching={files.isLoading}
loaderBackgroundBlur={5}
loaderVariant='dots'
+9 -1
View File
@@ -37,9 +37,17 @@ export default function FilePagation({ disableMediaPreview, exifEnabled, queryPa
})();
}, [page]);
const pages = usePaginatedFiles(page, !checked ? 'media' : null);
const pages = usePaginatedFiles(page, {
filter: !checked ? 'media' : 'none',
});
if (pages.isSuccess && pages.data.length === 0) {
if (page > 1 && numPages > 0) {
setPage(page - 1);
return null;
}
return (
<Center sx={{ flexDirection: 'column' }}>
<Group>
@@ -0,0 +1,118 @@
import { Button, Modal, Title, Tooltip } from '@mantine/core';
import { IconTrash } from '@tabler/icons-react';
import AnchorNext from 'components/AnchorNext';
import MutedText from 'components/MutedText';
import useFetch from 'hooks/useFetch';
import { DataTable } from 'mantine-datatable';
import { useEffect, useState } from 'react';
export type PendingFiles = {
id: number;
createdAt: string;
status: string;
chunks: number;
chunksComplete: number;
userId: number;
data: {
file: {
filename: string;
mimetype: string;
lastchunk: boolean;
identifier: string;
totalBytes: number;
};
code?: number;
message?: string;
};
};
export default function PendingFilesModal({ open, onClose }) {
const [incFiles, setIncFiles] = useState<PendingFiles[]>([]);
const [loading, setLoading] = useState(true);
const [selectedFiles, setSelectedFiles] = useState<PendingFiles[]>([]);
async function updateIncFiles() {
setLoading(true);
const files = await useFetch('/api/user/pending');
setIncFiles(files);
setLoading(false);
}
async function deleteIncFiles() {
await useFetch('/api/user/pending', 'DELETE', {
id: selectedFiles.map((file) => file.id),
});
updateIncFiles();
setSelectedFiles([]);
}
useEffect(() => {
updateIncFiles();
}, []);
useEffect(() => {
const interval = setInterval(() => {
if (open) updateIncFiles();
}, 5000);
return () => clearInterval(interval);
}, [open]);
return (
<Modal title={<Title>Pending Files</Title>} size='auto' opened={open} onClose={onClose}>
<MutedText size='xs'>Refreshing every 5 seconds...</MutedText>
<DataTable
withBorder
borderRadius='md'
highlightOnHover
verticalSpacing='sm'
minHeight={200}
records={incFiles ?? []}
columns={[
{ accessor: 'id', title: 'ID' },
{ accessor: 'createdAt', render: (file) => new Date(file.createdAt).toLocaleString() },
{ accessor: 'status', render: (file) => file.status.toLowerCase() },
{
accessor: 'progress',
title: 'Progress',
render: (file) => `${file.chunksComplete}/${file.chunks} chunks`,
},
{
accessor: 'message',
render: (file) =>
file.data.code === 200 ? (
<AnchorNext href={file.data.message} target='_blank'>
view file
</AnchorNext>
) : (
file.data.message
),
},
]}
fetching={loading}
loaderBackgroundBlur={5}
loaderVariant='dots'
onSelectedRecordsChange={setSelectedFiles}
selectedRecords={selectedFiles}
/>
{selectedFiles.length ? (
<Tooltip label='Clearing pending files will still leave the final file on the server.'>
<Button
variant='filled'
my='md'
color='red'
onClick={deleteIncFiles}
leftIcon={<IconTrash size='1rem' />}
fullWidth
>
Clear {selectedFiles.length} pending file{selectedFiles.length > 1 ? 's' : ''}
</Button>
</Tooltip>
) : null}
</Modal>
);
}
+17 -3
View File
@@ -1,16 +1,22 @@
import { Accordion, ActionIcon, Box, Group, Pagination, SimpleGrid, Title } from '@mantine/core';
import { IconFileUpload } from '@tabler/icons-react';
import { Accordion, ActionIcon, Box, Group, Pagination, SimpleGrid, Title, Tooltip } from '@mantine/core';
import { IconFileUpload, IconPhotoUp } from '@tabler/icons-react';
import File from 'components/File';
import useFetch from 'hooks/useFetch';
import { usePaginatedFiles } from 'lib/queries/files';
import Link from 'next/link';
import { useEffect, useState } from 'react';
import FilePagation from './FilePagation';
import PendingFilesModal from './PendingFilesModal';
export default function Files({ disableMediaPreview, exifEnabled, queryPage, compress }) {
const [favoritePage, setFavoritePage] = useState(1);
const [favoriteNumPages, setFavoriteNumPages] = useState(0);
const favoritePages = usePaginatedFiles(favoritePage, 'media', true);
const favoritePages = usePaginatedFiles(favoritePage, {
filter: 'media',
favorite: true,
});
const [open, setOpen] = useState(false);
useEffect(() => {
(async () => {
@@ -21,11 +27,19 @@ export default function Files({ disableMediaPreview, exifEnabled, queryPage, com
return (
<>
<PendingFilesModal open={open} onClose={() => setOpen(false)} />
<Group mb='md'>
<Title>Files</Title>
<ActionIcon component={Link} href='/dashboard/upload/file' variant='filled' color='primary'>
<IconFileUpload size='1rem' />
</ActionIcon>
<Tooltip label='View pending uploads'>
<ActionIcon onClick={() => setOpen(true)} variant='filled' color='primary'>
<IconPhotoUp size='1rem' />
</ActionIcon>
</Tooltip>
</Group>
{favoritePages.isSuccess && favoritePages.data.length ? (
<Accordion
+13 -20
View File
@@ -112,7 +112,7 @@ export default function Folders({ disableMediaPreview, exifEnabled, compress })
const makePublic = async (folder) => {
const res = await useFetch(`/api/user/folders/${folder.id}`, 'PATCH', {
public: folder.public ? false : true,
public: !folder.public,
});
if (!res.error) {
@@ -363,25 +363,18 @@ export default function Folders({ disableMediaPreview, exifEnabled, compress })
aria-label='copy link'
onClick={() => {
clipboard.copy(`${window.location.origin}/folder/${folder.id}`);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied folder link',
message: (
<>
Copied{' '}
<AnchorNext href={`/folder/${folder.id}`}>folder link</AnchorNext> to
clipboard
</>
),
color: 'green',
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied folder link',
message: (
<>
Copied <AnchorNext href={`/folder/${folder.id}`}>folder link</AnchorNext>{' '}
to clipboard
</>
),
color: 'green',
icon: <IconClipboardCopy size='1rem' />,
});
}}
>
<IconClipboardCopy size='1rem' />
+13 -32
View File
@@ -30,18 +30,18 @@ import {
import MutedText from 'components/MutedText';
import useFetch from 'hooks/useFetch';
import { listViewInvitesSelector } from 'lib/recoil/settings';
import { expireText, relativeTime } from 'lib/utils/client';
import { expireReadToDate, expireText, relativeTime } from 'lib/utils/client';
import { DataTable, DataTableSortStatus } from 'mantine-datatable';
import { useRouter } from 'next/router';
import { useEffect, useState } from 'react';
import { useRecoilState } from 'recoil';
const expires = ['30m', '1h', '6h', '12h', '1d', '3d', '5d', '7d', 'never'];
const expires = ['30min', '1h', '6h', '12h', '1d', '3d', '5d', '7d', 'never'];
function CreateInviteModal({ open, setOpen, updateInvites }) {
const form = useForm({
initialValues: {
expires: '30m',
expires: '30min',
count: 1,
},
});
@@ -50,26 +50,12 @@ function CreateInviteModal({ open, setOpen, updateInvites }) {
if (!expires.includes(values.expires)) return form.setFieldError('expires', 'Invalid expiration');
if (values.count < 1 || values.count > 100)
return form.setFieldError('count', 'Must be between 1 and 100');
const expiresAt =
values.expires === 'never'
? null
: new Date(
{
'30m': Date.now() + 30 * 60 * 1000,
'1h': Date.now() + 60 * 60 * 1000,
'6h': Date.now() + 6 * 60 * 60 * 1000,
'12h': Date.now() + 12 * 60 * 60 * 1000,
'1d': Date.now() + 24 * 60 * 60 * 1000,
'3d': Date.now() + 3 * 24 * 60 * 60 * 1000,
'5d': Date.now() + 5 * 24 * 60 * 60 * 1000,
'7d': Date.now() + 7 * 24 * 60 * 60 * 1000,
}[values.expires]
);
const expiresAt = values.expires === 'never' ? null : expireReadToDate(values.expires);
setOpen(false);
const res = await useFetch('/api/auth/invite', 'POST', {
expiresAt,
expiresAt: expiresAt === null ? null : `date=${expiresAt.toISOString()}`,
count: values.count,
});
@@ -99,8 +85,9 @@ function CreateInviteModal({ open, setOpen, updateInvites }) {
label='Expires'
id='expires'
{...form.getInputProps('expires')}
maxDropdownHeight={100}
data={[
{ value: '30m', label: '30 minutes' },
{ value: '30min', label: '30 minutes' },
{ value: '1h', label: '1 hour' },
{ value: '6h', label: '6 hours' },
{ value: '12h', label: '12 hours' },
@@ -197,18 +184,12 @@ export default function Invites() {
const handleCopy = async (invite) => {
clipboard.copy(`${window.location.protocol}//${window.location.host}/auth/register?code=${invite.code}`);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
};
const updateInvites = async () => {
+7 -2
View File
@@ -3,8 +3,10 @@ import { closeAllModals, openConfirmModal } from '@mantine/modals';
import { showNotification, updateNotification } from '@mantine/notifications';
import { IconFiles, IconFilesOff } from '@tabler/icons-react';
import useFetch from 'hooks/useFetch';
import { useState } from 'react';
export default function ClearStorage({ open, setOpen, check, setCheck }) {
export default function ClearStorage({ open, setOpen }) {
const [check, setCheck] = useState(false);
const handleDelete = async (datasource: boolean, orphaned?: boolean) => {
showNotification({
id: 'clear-uploads',
@@ -38,7 +40,10 @@ export default function ClearStorage({ open, setOpen, check, setCheck }) {
return (
<Modal
opened={open}
onClose={() => setOpen(false)}
onClose={() => {
setOpen(false);
setCheck(() => false);
}}
title={<Title size='sm'>Are you sure you want to clear all uploads in the database?</Title>}
>
<Checkbox
+3 -5
View File
@@ -4,7 +4,7 @@ import { Icon2fa, IconBarcodeOff, IconCheck } from '@tabler/icons-react';
import useFetch from 'hooks/useFetch';
import { useEffect, useState } from 'react';
export function TotpModal({ opened, onClose, deleteTotp, setTotpEnabled }) {
export function TotpModal({ opened, onClose, deleteTotp, setUser }) {
const [secret, setSecret] = useState('');
const [qrCode, setQrCode] = useState('');
const [disabled, setDisabled] = useState(false);
@@ -52,8 +52,7 @@ export function TotpModal({ opened, onClose, deleteTotp, setTotpEnabled }) {
icon: <Icon2fa size='1rem' />,
});
setTotpEnabled(false);
setUser((user) => ({ ...user, totpSecret: null }));
onClose();
}
@@ -83,8 +82,7 @@ export function TotpModal({ opened, onClose, deleteTotp, setTotpEnabled }) {
icon: <Icon2fa size='1rem' />,
});
setTotpEnabled(true);
setUser((user) => ({ ...user, totpSecret: secret }));
onClose();
}
+28 -4
View File
@@ -1,9 +1,11 @@
import {
ActionIcon,
Anchor,
Box,
Button,
Card,
ColorInput,
CopyButton,
FileInput,
Group,
Image,
@@ -23,6 +25,8 @@ import {
IconBrandDiscordFilled,
IconBrandGithubFilled,
IconBrandGoogle,
IconCheck,
IconClipboardCopy,
IconFileExport,
IconFiles,
IconFilesOff,
@@ -89,7 +93,7 @@ export default function Manage({ oauth_registration, oauth_providers: raw_oauth_
const [file, setFile] = useState<File | null>(null);
const [fileDataURL, setFileDataURL] = useState(user.avatar ?? null);
const [totpEnabled, setTotpEnabled] = useState(!!user.totpSecret);
const [checked, setCheck] = useState(false);
const [tokenShown, setTokenShown] = useState(false);
const getDataURL = (f: File): Promise<string> => {
return new Promise((res, rej) => {
@@ -355,7 +359,8 @@ export default function Manage({ oauth_registration, oauth_providers: raw_oauth_
useEffect(() => {
getExports();
interval.start();
}, [totpEnabled]);
setTotpEnabled(() => !!user.totpSecret);
}, [user]);
return (
<>
@@ -365,6 +370,25 @@ export default function Manage({ oauth_registration, oauth_providers: raw_oauth_
<AnchorNext href='https://zipline.diced.tech/docs/guides/variables'>the docs</AnchorNext> for
variables
</MutedText>
<TextInput
rightSection={
<CopyButton value={user.token} timeout={1000}>
{({ copied, copy }) => (
<ActionIcon onClick={copy}>
{copied ? <IconCheck color='green' size='1rem' /> : <IconClipboardCopy size='1rem' />}
</ActionIcon>
)}
</CopyButton>
}
// @ts-ignore (this works even though ts doesn't allow for it)
component='span'
label='Token'
onClick={() => setTokenShown(true)}
>
{tokenShown ? user.token : '[click to reveal]'}
</TextInput>
<form onSubmit={form.onSubmit((v) => onSubmit(v))}>
<TextInput id='username' label='Username' my='sm' {...form.getInputProps('username')} />
<PasswordInput
@@ -450,7 +474,7 @@ export default function Manage({ oauth_registration, oauth_providers: raw_oauth_
opened={totpOpen}
onClose={() => setTotpOpen(false)}
deleteTotp={totpEnabled}
setTotpEnabled={setTotpEnabled}
setUser={setUser}
/>
</Box>
)}
@@ -626,7 +650,7 @@ export default function Manage({ oauth_registration, oauth_providers: raw_oauth_
<ShareX user={user} open={shareXOpen} setOpen={setShareXOpen} />
<Flameshot user={user} open={flameshotOpen} setOpen={setFlameshotOpen} />
<ClearStorage open={clrStorOpen} setOpen={setClrStorOpen} check={checked} setCheck={setCheck} />
<ClearStorage open={clrStorOpen} setOpen={setClrStorOpen} />
</>
);
}
+6 -12
View File
@@ -27,18 +27,12 @@ export default function MetadataView({ fileId }) {
const copy = (value) => {
clipboard.copy(value);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: value,
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: value,
icon: <IconClipboardCopy size='1rem' />,
});
};
const searchValue = (value) => {
+34 -26
View File
@@ -13,8 +13,11 @@ import { useEffect, useState } from 'react';
import { useRecoilValue } from 'recoil';
import showFilesModal from './showFilesModal';
import useUploadOptions from './useUploadOptions';
import { useRouter } from 'next/router';
export default function File({ chunks: chunks_config }) {
const router = useRouter();
const clipboard = useClipboard();
const modals = useModals();
const user = useRecoilValue(userSelector);
@@ -25,6 +28,24 @@ export default function File({ chunks: chunks_config }) {
const [options, setOpened, OptionsModal] = useUploadOptions();
const beforeUnload = (e: BeforeUnloadEvent) => {
if (loading) {
e.preventDefault();
e.returnValue = "Are you sure you want to leave? Your upload(s) won't be saved.";
return e.returnValue;
}
};
const beforeRouteChange = (url: string) => {
if (loading) {
const confirmed = confirm("Are you sure you want to leave? Your upload(s) won't be saved.");
if (!confirmed) {
router.events.emit('routeChangeComplete', url);
throw 'Route change aborted';
}
}
};
useEffect(() => {
const listener = (e: ClipboardEvent) => {
const item = Array.from(e.clipboardData.items).find((x) => /^image/.test(x.type));
@@ -41,8 +62,14 @@ export default function File({ chunks: chunks_config }) {
};
document.addEventListener('paste', listener);
return () => document.removeEventListener('paste', listener);
}, []);
window.addEventListener('beforeunload', beforeUnload);
router.events.on('routeChangeStart', beforeRouteChange);
return () => {
window.removeEventListener('beforeunload', beforeUnload);
router.events.off('routeChangeStart', beforeRouteChange);
document.removeEventListener('paste', listener);
};
}, [loading, beforeUnload, beforeRouteChange]);
const handleChunkedFiles = async (expiresAt: Date, toChunkFiles: File[]) => {
for (let i = 0; i !== toChunkFiles.length; ++i) {
@@ -71,18 +98,6 @@ export default function File({ chunks: chunks_config }) {
await new Promise((resolve) => setTimeout(resolve, 100));
}
// if last chunk send notif that it will take a while
if (j === chunks.length - 1) {
updateNotification({
id: 'upload-chunked',
title: 'Finalizing partial upload',
message: 'This may take a while...',
icon: <IconFileTime size='1rem' />,
color: 'yellow',
autoClose: false,
});
}
const body = new FormData();
body.append('file', chunks[j].blob);
@@ -109,25 +124,18 @@ export default function File({ chunks: chunks_config }) {
if (j === chunks.length - 1) {
updateNotification({
id: 'upload-chunked',
title: 'Upload Successful',
message: '',
title: 'Finalizing partial upload',
message:
'The upload has been offloaded, and will complete in the background. You can see processing files in the files tab.',
icon: <IconFileTime size='1rem' />,
color: 'green',
icon: <IconFileUpload size='1rem' />,
autoClose: true,
});
showFilesModal(clipboard, modals, json.files);
invalidateFiles();
setFiles([]);
setProgress(100);
setTimeout(() => setProgress(0), 1000);
clipboard.copy(json.files[0]);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
}
ready = true;
+6 -12
View File
@@ -7,18 +7,12 @@ export default function showFilesModal(clipboard, modals, files: string[]) {
const open = (idx: number) => window.open(files[idx], '_blank');
const copy = (idx: number) => {
clipboard.copy(files[idx]);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: <AnchorNext href={files[idx]}>{files[idx]}</AnchorNext>,
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: <AnchorNext href={files[idx]}>{files[idx]}</AnchorNext>,
icon: <IconClipboardCopy size='1rem' />,
});
};
modals.openModal({
+6 -12
View File
@@ -169,18 +169,12 @@ export default function Urls() {
const copyURL = (u) => {
clipboard.copy(`${window.location.protocol}//${window.location.host}${u.url}`);
if (!navigator.clipboard)
showNotification({
title: 'Unable to copy to clipboard',
message: 'Zipline is unable to copy to clipboard due to security reasons.',
color: 'red',
});
else
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
showNotification({
title: 'Copied to clipboard',
message: '',
icon: <IconClipboardCopy size='1rem' />,
});
};
const urlDelete = useURLDelete();
+82
View File
@@ -0,0 +1,82 @@
import { ActionIcon, Button, Center, Group, SimpleGrid, Title } from '@mantine/core';
import { File } from '@prisma/client';
import { IconArrowLeft, IconFile } from '@tabler/icons-react';
import FileComponent from 'components/File';
import MutedText from 'components/MutedText';
import useFetch from 'hooks/useFetch';
import { userSelector } from 'lib/recoil/user';
import { useRouter } from 'next/router';
import { useEffect, useState } from 'react';
import { useRecoilState } from 'recoil';
type UserFiles = {
id: number;
username: string;
files?: File[];
error?: unknown;
};
export default function UserFiles({ userId, disableMediaPreview, exifEnabled, compress }) {
const [currentUser, viewUser] = useState<UserFiles>({ id: 0, username: 'user' });
const [self] = useRecoilState(userSelector);
const { push } = useRouter();
useEffect(() => {
if (self.id == userId) push('/dashboard/files');
(async () => {
const user: UserFiles = await useFetch(`/api/user/${userId}`);
if (!user.error) {
viewUser(user);
} else {
push('/dashboard');
}
})();
}, [userId]);
if (!currentUser.files || currentUser.files.length === 0) {
return (
<Center sx={{ flexDirection: 'column' }}>
<Group>
<div>
<IconFile size={48} />
</div>
<div>
<Title>Nothing here</Title>
<MutedText size='md'>
{currentUser.username} seems to have not uploaded any files... yet
</MutedText>
</div>
<Button size='md' onClick={() => push('/dashboard/users')}>
Head back?
</Button>
</Group>
</Center>
);
}
return (
<>
<Group mb='md'>
<ActionIcon size='lg' onClick={() => push('/dashboard/users')} color='primary'>
<IconArrowLeft />
</ActionIcon>
<Title>{currentUser.username}&apos;s Files</Title>
</Group>
<SimpleGrid cols={3} spacing='lg' breakpoints={[{ maxWidth: 'sm', cols: 1, spacing: 'sm' }]}>
{currentUser.files.map((file) => (
<div key={file.id}>
<FileComponent
image={file}
disableMediaPreview={disableMediaPreview}
exifEnabled={exifEnabled}
onDash={compress}
otherUser={true}
/>
</div>
))}
</SimpleGrid>
</>
);
}
+12
View File
@@ -6,6 +6,7 @@ import type { User } from '@prisma/client';
import {
IconClipboardCopy,
IconEdit,
IconExternalLink,
IconGridDots,
IconList,
IconUserExclamation,
@@ -116,6 +117,10 @@ export default function Users() {
}
};
const openUser = async (user) => {
await router.push(`/dashboard/users/${user.id}`);
};
useEffect(() => {
updateUsers();
}, []);
@@ -181,6 +186,13 @@ export default function Users() {
<IconEdit size='1rem' />
</ActionIcon>
</Tooltip>
{(!self.superAdmin && user.superAdmin) || (self.superAdmin && user.superAdmin) ? null : (
<Tooltip label='Open user'>
<ActionIcon color='cyan' onClick={() => openUser(user)}>
<IconExternalLink size='1rem' />
</ActionIcon>
</Tooltip>
)}
</Group>
),
},
+7
View File
@@ -1,5 +1,6 @@
export interface ConfigCore {
return_https: boolean;
temp_directory: string;
secret: string;
host: string;
port: number;
@@ -9,6 +10,7 @@ export interface ConfigCore {
stats_interval: number;
invites_interval: number;
thumbnails_interval: number;
}
export interface ConfigCompression {
@@ -119,9 +121,13 @@ export interface ConfigFeatures {
headless: boolean;
default_avatar: string;
robots_txt: string;
}
export interface ConfigOAuth {
bypass_local_login: boolean;
github_client_id?: string;
github_client_secret?: string;
@@ -135,6 +141,7 @@ export interface ConfigOAuth {
export interface ConfigChunks {
max_size: number;
chunks_size: number;
enabled: boolean;
}
export interface ConfigMfa {
+9
View File
@@ -57,13 +57,17 @@ export default function readConfig() {
const maps = [
map('CORE_RETURN_HTTPS', 'boolean', 'core.return_https'),
map('CORE_TEMP_DIRECTORY', 'path', 'core.temp_directory'),
map('CORE_SECRET', 'string', 'core.secret'),
map('CORE_HOST', 'string', 'core.host'),
map('CORE_PORT', 'number', 'core.port'),
map('CORE_DATABASE_URL', 'string', 'core.database_url'),
map('CORE_LOGGER', 'boolean', 'core.logger'),
map('CORE_STATS_INTERVAL', 'number', 'core.stats_interval'),
map('CORE_INVITES_INTERVAL', 'number', 'core.invites_interval'),
map('CORE_THUMBNAILS_INTERVAL', 'number', 'core.thumbnails_interval'),
map('CORE_COMPRESSION_ENABLED', 'boolean', 'core.compression.enabled'),
map('CORE_COMPRESSION_THRESHOLD', 'human-to-byte', 'core.compression.threshold'),
map('CORE_COMPRESSION_ON_DASHBOARD', 'boolean', 'core.compression.on_dashboard'),
@@ -135,6 +139,8 @@ export default function readConfig() {
map('DISCORD_SHORTEN_EMBED_THUMBNAIL', 'boolean', 'discord.shorten.embed.thumbnail'),
map('DISCORD_SHORTEN_EMBED_TIMESTAMP', 'boolean', 'discord.shorten.embed.timestamp'),
map('OAUTH_BYPASS_LOCAL_LOGIN', 'boolean', 'oauth.bypass_local_login'),
map('OAUTH_GITHUB_CLIENT_ID', 'string', 'oauth.github_client_id'),
map('OAUTH_GITHUB_CLIENT_SECRET', 'string', 'oauth.github_client_secret'),
@@ -155,8 +161,11 @@ export default function readConfig() {
map('FEATURES_DEFAULT_AVATAR', 'path', 'features.default_avatar'),
map('FEATURES_ROBOTS_TXT', 'boolean', 'features.robots_txt'),
map('CHUNKS_MAX_SIZE', 'human-to-byte', 'chunks.max_size'),
map('CHUNKS_CHUNKS_SIZE', 'human-to-byte', 'chunks.chunks_size'),
map('CHUNKS_ENABLED', 'boolean', 'chunks.enabled'),
map('MFA_TOTP_ISSUER', 'string', 'mfa.totp_issuer'),
map('MFA_TOTP_ENABLED', 'boolean', 'mfa.totp_enabled'),
+13 -3
View File
@@ -3,6 +3,8 @@ import type { Config } from './Config';
import { inspect } from 'util';
import Logger from 'lib/logger';
import { humanToBytes } from 'utils/bytes';
import { tmpdir } from 'os';
import { join, resolve } from 'path';
const discord_content = s
.object({
@@ -27,13 +29,15 @@ const discord_content = s
const validator = s.object({
core: s.object({
return_https: s.boolean.default(false),
temp_directory: s.string.default(join(tmpdir(), 'zipline')),
secret: s.string.lengthGreaterThanOrEqual(8),
host: s.string.default('0.0.0.0'),
port: s.number.default(3000),
database_url: s.string,
logger: s.boolean.default(false),
stats_interval: s.number.default(1800),
invites_interval: s.number.default(1800),
stats_interval: s.number.default(1800), // 30m
invites_interval: s.number.default(1800), // 30m
thumbnails_interval: s.number.default(600), // 10m
compression: s
.object({
enabled: s.boolean.default(false),
@@ -50,7 +54,7 @@ const validator = s.object({
type: s.enum('local', 's3', 'supabase').default('local'),
local: s
.object({
directory: s.string.default('./uploads'),
directory: s.string.default(resolve('./uploads')).transform((v) => resolve(v)),
})
.default({
directory: './uploads',
@@ -165,6 +169,8 @@ const validator = s.object({
.nullish.default(null),
oauth: s
.object({
bypass_local_login: s.boolean.default(false),
github_client_id: s.string.nullable.default(null),
github_client_secret: s.string.nullable.default(null),
@@ -184,6 +190,7 @@ const validator = s.object({
user_registration: s.boolean.default(false),
headless: s.boolean.default(false),
default_avatar: s.string.nullable.default(null),
robots_txt: s.boolean.default(false),
})
.default({
invites: false,
@@ -193,15 +200,18 @@ const validator = s.object({
user_registration: false,
headless: false,
default_avatar: null,
robots_txt: false,
}),
chunks: s
.object({
max_size: s.number.default(humanToBytes('90MB')),
chunks_size: s.number.default(humanToBytes('20MB')),
enabled: s.boolean.default(true),
})
.default({
max_size: humanToBytes('90MB'),
chunks_size: humanToBytes('20MB'),
enabled: true,
}),
mfa: s
.object({
+9 -6
View File
@@ -11,22 +11,23 @@ export class Local extends Datasource {
}
public async save(file: string, data: Buffer): Promise<void> {
await writeFile(join(process.cwd(), this.path, file), data);
await writeFile(join(this.path, file), data);
}
public async delete(file: string): Promise<void> {
await rm(join(process.cwd(), this.path, file));
await rm(join(this.path, file));
}
public async clear(): Promise<void> {
const files = await readdir(join(process.cwd(), this.path));
const files = await readdir(this.path);
for (let i = 0; i !== files.length; ++i) {
await rm(join(process.cwd(), this.path, files[i]));
await rm(join(this.path, files[i]));
}
}
public get(file: string): ReadStream {
const full = join(process.cwd(), this.path, file);
const full = join(this.path, file);
if (!existsSync(full)) return null;
try {
@@ -37,7 +38,9 @@ export class Local extends Datasource {
}
public async size(file: string): Promise<number> {
const stats = await stat(join(process.cwd(), this.path, file));
const full = join(this.path, file);
if (!existsSync(full)) return 0;
const stats = await stat(full);
return stats.size;
}
+4 -4
View File
@@ -50,22 +50,22 @@ export class S3 extends Datasource {
}
public size(file: string): Promise<number> {
return new Promise((res, rej) => {
return new Promise((res) => {
this.s3.statObject(this.config.bucket, file, (err, stat) => {
if (err) rej(err);
if (err) res(0);
else res(stat.size);
});
});
}
public async fullSize(): Promise<number> {
return new Promise((res, rej) => {
return new Promise((res) => {
const objects = this.s3.listObjectsV2(this.config.bucket, '', true);
let size = 0;
objects.on('data', (item) => (size += item.size));
objects.on('end', (err) => {
if (err) rej(err);
if (err) res(0);
else res(size);
});
});
+2 -2
View File
@@ -63,13 +63,13 @@ export async function sendUpload(user: User, file: File, raw_link: string, link:
thumbnail:
isImage && parsed.embed.thumbnail
? {
url: parsed.url,
url: raw_link,
}
: null,
image:
isImage && parsed.embed.image
? {
url: parsed.url,
url: raw_link,
}
: null,
},
+4 -1
View File
@@ -2,6 +2,7 @@ import date from './date';
import gfycat from './gfycat';
import random from './random';
import uuid from './uuid';
import { parse } from 'path';
export type NameFormat = 'random' | 'date' | 'uuid' | 'name' | 'gfycat';
export const NameFormats: NameFormat[] = ['random', 'date', 'uuid', 'name', 'gfycat'];
@@ -14,7 +15,9 @@ export default async function formatFileName(nameFormat: NameFormat, originalNam
case 'uuid':
return uuid();
case 'name':
return originalName.split('.')[0];
const { name } = parse(originalName);
return name;
case 'gfycat':
return gfycat();
default:
+2
View File
@@ -16,6 +16,7 @@ export type ServerSideProps = {
user_registration: boolean;
oauth_registration: boolean;
oauth_providers: string;
bypass_local_login: boolean;
chunks_size: number;
max_size: number;
totp_enabled: boolean;
@@ -60,6 +61,7 @@ export const getServerSideProps: GetServerSideProps<ServerSideProps> = async (ct
user_registration: config.features.user_registration,
oauth_registration: config.features.oauth_registration,
oauth_providers: JSON.stringify(oauth_providers),
bypass_local_login: config.oauth?.bypass_local_login ?? false,
chunks_size: config.chunks.chunks_size,
max_size: config.chunks.max_size,
totp_enabled: config.mfa.totp_enabled,
+3 -3
View File
@@ -135,7 +135,7 @@ export const withOAuth =
} else throw e;
}
res.setUserCookie(user.id);
res.setUserCookie(user.uuid);
logger.info(`User ${user.username} (${user.id}) linked account via oauth(${provider})`);
return res.redirect('/');
@@ -153,7 +153,7 @@ export const withOAuth =
},
});
res.setUserCookie(user.id);
res.setUserCookie(user.uuid);
logger.info(`User ${user.username} (${user.id}) logged in via oauth(${provider})`);
return res.redirect('/dashboard');
@@ -203,7 +203,7 @@ export const withOAuth =
logger.debug(`created user ${JSON.stringify(nuser)} via oauth(${provider})`);
logger.info(`Created user ${nuser.username} via oauth(${provider})`);
res.setUserCookie(nuser.id);
res.setUserCookie(nuser.uuid);
logger.info(`User ${nuser.username} (${nuser.id}) logged in via oauth(${provider})`);
return res.redirect('/dashboard');
+6 -6
View File
@@ -54,7 +54,7 @@ export type NextApiRes = NextApiResponse &
NextApiResExtraObj & {
json: (json: Record<string, unknown>, status?: number) => void;
setCookie: (name: string, value: unknown, options: CookieSerializeOptions) => void;
setUserCookie: (id: number) => void;
setUserCookie: (id: string) => void;
};
export type ZiplineApiConfig = {
@@ -184,7 +184,7 @@ export const withZipline =
const user = await prisma.user.findFirst({
where: {
id: Number(userId),
uuid: userId,
},
include: {
oauth: true,
@@ -202,22 +202,22 @@ export const withZipline =
}
};
res.setCookie = (name: string, value: unknown, options: CookieSerializeOptions = {}) => {
res.setCookie = (name: string, value: string, options: CookieSerializeOptions = {}) => {
if ('maxAge' in options) {
options.expires = new Date(Date.now() + options.maxAge * 1000);
options.maxAge /= 1000;
}
const signed = sign64(String(value), config.core.secret);
const signed = sign64(value, config.core.secret);
Logger.get('api').debug(`headers(${JSON.stringify(req.headers)}): cookie(${name}, ${value})`);
res.setHeader('Set-Cookie', serialize(name, signed, options));
};
res.setUserCookie = (id: number) => {
res.setUserCookie = (id: string) => {
req.cleanCookie('user');
res.setCookie('user', String(id), {
res.setCookie('user', id, {
sameSite: 'lax',
expires: new Date(Date.now() + 6.048e8 * 2),
path: '/',
+16 -6
View File
@@ -33,13 +33,23 @@ export const useFiles = (query: { [key: string]: string } = {}) => {
);
});
};
export const usePaginatedFiles = (page?: number, filter = 'media', favorite = null) => {
const queryBuilder = new URLSearchParams({
export type PaginatedFilesOptions = {
filter: 'media' | 'none';
favorite: boolean;
sortBy: 'createdAt' | 'views' | 'expiresAt' | 'size' | 'name' | 'mimetype';
order: 'asc' | 'desc';
};
export const usePaginatedFiles = (page?: number, options?: Partial<PaginatedFilesOptions>) => {
const queryString = new URLSearchParams({
page: Number(page || '1').toString(),
filter,
...(favorite !== null && { favorite: favorite.toString() }),
});
const queryString = queryBuilder.toString();
filter: options?.filter ?? 'none',
// ...(options?.favorite !== null && { favorite: options?.favorite?.toString() }),
favorite: options.favorite ? 'true' : '',
sortBy: options.sortBy ?? '',
order: options.order ?? '',
}).toString();
return useQuery<UserFilesResponse[]>(['files', queryString], async () => {
return fetch('/api/user/paged?' + queryString)
+1 -2
View File
@@ -3,7 +3,6 @@ import { createWriteStream } from 'fs';
import { ExifTool, Tags } from 'exiftool-vendored';
import datasource from 'lib/datasource';
import Logger from 'lib/logger';
import { tmpdir } from 'os';
import { join } from 'path';
import { readFile, unlink } from 'fs/promises';
@@ -34,7 +33,7 @@ export async function readMetadata(filePath: string): Promise<Tags> {
export async function removeGPSData(image: File): Promise<void> {
const exiftool = new ExifTool({ cleanupChildProcs: false });
const file = join(tmpdir(), `zipline-exif-remove-${Date.now()}-${image.name}`);
const file = join(config.core.temp_directory, `zipline-exif-remove-${Date.now()}-${image.name}`);
logger.debug(`writing temp file to remove GPS data: ${file}`);
const stream = await datasource.get(image.name);
+5 -2
View File
@@ -11,7 +11,10 @@ export type ParseValue = {
export function parseString(str: string, value: ParseValue) {
if (!str) return null;
str = str.replace(/\{link\}/gi, value.link).replace(/\{raw_link\}/gi, value.raw_link);
str = str
.replace(/\{link\}/gi, value.link)
.replace(/\{raw_link\}/gi, value.raw_link)
.replace(/\\n/g, '\n');
const re = /\{(?<type>file|url|user)\.(?<prop>\w+)(::(?<mod>\w+))?\}/gi;
let matches: RegExpMatchArray;
@@ -24,7 +27,7 @@ export function parseString(str: string, value: ParseValue) {
continue;
}
if (['password', 'avatar'].includes(matches.groups.prop)) {
if (['password', 'avatar', 'uuid'].includes(matches.groups.prop)) {
str = replaceCharsFromString(str, '{unknown_property}', matches.index, re.lastIndex);
re.lastIndex = matches.index;
continue;
+9 -3
View File
@@ -2,8 +2,10 @@ import { Button, Stack, Title, Tooltip } from '@mantine/core';
import MutedText from 'components/MutedText';
import Head from 'next/head';
import Link from 'next/link';
import { useRouter } from 'next/router';
export default function FiveHundred() {
const { asPath } = useRouter();
return (
<>
<Head>
@@ -24,9 +26,13 @@ export default function FiveHundred() {
<Tooltip label={"Take a look at Zipline's logs and the browser console for more info"}>
<MutedText>Internal server error</MutedText>
</Tooltip>
<Button component={Link} href='/dashboard'>
Head to the Dashboard
</Button>
{asPath === '/dashboard' ? (
<Button onClick={() => window.location.reload()}>Attempt Refresh</Button>
) : (
<Button component={Link} href='/dashboard'>
Head to the Dashboard
</Button>
)}
</Stack>
</>
);
+1 -1
View File
@@ -6,7 +6,7 @@ const logger = Logger.get('admin');
async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
try {
const { datasource, orphaned } = req.body;
const { orphaned } = req.body;
if (orphaned) {
const { count } = await prisma.file.deleteMany({
where: {
+1
View File
@@ -7,6 +7,7 @@ import { extname } from 'path';
async function handler(req: NextApiReq, res: NextApiRes) {
const { id, password } = req.query;
if (isNaN(Number(id))) return res.badRequest('invalid id');
const file = await prisma.file.findFirst({
where: {
+3 -5
View File
@@ -3,6 +3,7 @@ import Logger from 'lib/logger';
import { NextApiReq, NextApiRes, UserExtended, withZipline } from 'lib/middleware/withZipline';
import prisma from 'lib/prisma';
import { randomChars } from 'lib/util';
import { parseExpiry } from 'lib/utils/client';
const logger = Logger.get('invite');
@@ -15,11 +16,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
count: number;
};
const expiry = expiresAt ? new Date(expiresAt) : null;
if (expiry) {
if (!expiry.getTime()) return res.badRequest('invalid date');
if (expiry.getTime() < Date.now()) return res.badRequest('date is in the past');
}
const expiry = parseExpiry(expiresAt);
if (!expiry) return res.badRequest('invalid date');
const counts = count ? count : 1;
if (counts > 1) {
+1 -1
View File
@@ -56,7 +56,7 @@ async function handler(req: NextApiReq, res: NextApiRes) {
if (!success) return res.badRequest('Invalid code', { totp: true });
}
res.setUserCookie(user.id);
res.setUserCookie(user.uuid);
logger.info(`User ${user.username} (${user.id}) logged in`);
return res.json({ success: true });
+1 -2
View File
@@ -6,7 +6,6 @@ import Logger from 'lib/logger';
import prisma from 'lib/prisma';
import { readMetadata } from 'lib/utils/exif';
import { NextApiReq, NextApiRes, UserExtended, withZipline } from 'middleware/withZipline';
import { tmpdir } from 'os';
import { join } from 'path';
const logger = Logger.get('exif');
@@ -41,7 +40,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json(data);
} else {
const file = join(tmpdir(), `zipline-exif-read-${Date.now()}-${image.name}`);
const file = join(config.core.temp_directory, `zipline-exif-read-${Date.now()}-${image.name}`);
logger.debug(`writing temp file to view metadata: ${file}`);
const stream = await datasource.get(image.name);
+32 -87
View File
@@ -1,5 +1,5 @@
import { InvisibleFile } from '@prisma/client';
import { readdir, readFile, unlink, writeFile } from 'fs/promises';
import { writeFile } from 'fs/promises';
import zconfig from 'lib/config';
import datasource from 'lib/datasource';
import { sendUpload } from 'lib/discord';
@@ -12,9 +12,9 @@ import { createInvisImage, hashPassword } from 'lib/util';
import { parseExpiry } from 'lib/utils/client';
import { removeGPSData } from 'lib/utils/exif';
import multer from 'multer';
import { tmpdir } from 'os';
import { join } from 'path';
import { join, parse } from 'path';
import sharp from 'sharp';
import { Worker } from 'worker_threads';
const uploader = multer();
const logger = Logger.get('upload');
@@ -79,7 +79,7 @@ async function handler(req: NextApiReq, res: NextApiRes) {
if (fileMaxViews < 0) return res.badRequest('invalid max views (max views < 0)');
// handle partial uploads before ratelimits
if (req.headers['content-range']) {
if (req.headers['content-range'] && zconfig.chunks.enabled) {
// parses content-range header (bytes start-end/total)
const [start, end, total] = req.headers['content-range']
.replace('bytes ', '')
@@ -104,94 +104,34 @@ async function handler(req: NextApiReq, res: NextApiRes) {
})}`
);
const tempFile = join(tmpdir(), `zipline_partial_${identifier}_${start}_${end}`);
const tempFile = join(zconfig.core.temp_directory, `zipline_partial_${identifier}_${start}_${end}`);
logger.debug(`writing partial to disk ${tempFile}`);
await writeFile(tempFile, req.files[0].buffer);
if (lastchunk) {
const partials = await readdir(tmpdir()).then((files) =>
files.filter((x) => x.startsWith(`zipline_partial_${identifier}`))
);
const readChunks = partials.map((x) => {
const [, , , start, end] = x.split('_');
return { start: Number(start), end: Number(end), filename: x };
});
// combine chunks
const chunks = new Uint8Array(total);
for (let i = 0; i !== readChunks.length; ++i) {
const chunkData = readChunks[i];
const buffer = await readFile(join(tmpdir(), chunkData.filename));
await unlink(join(tmpdir(), readChunks[i].filename));
chunks.set(buffer, chunkData.start);
}
const ext = filename.split('.').length === 1 ? '' : filename.split('.').pop();
if (zconfig.uploader.disabled_extensions.includes(ext))
return res.error('disabled extension recieved: ' + ext);
const fileName = await formatFileName(format, filename);
let password = null;
if (req.headers.password) {
password = await hashPassword(req.headers.password as string);
}
const compressionUsed = imageCompressionPercent && mimetype.startsWith('image/');
let invis: InvisibleFile;
const file = await prisma.file.create({
data: {
name: `${fileName}${compressionUsed ? '.jpg' : `${ext ? '.' : ''}${ext}`}`,
mimetype,
userId: user.id,
embed: !!req.headers.embed,
password,
expiresAt: expiry,
maxViews: fileMaxViews,
originalName: req.headers['original-name'] ? filename ?? null : null,
new Worker('./dist/worker/upload.js', {
workerData: {
user,
file: {
filename,
mimetype,
identifier,
lastchunk,
totalBytes: total,
},
response: {
expiresAt: expiry,
format,
imageCompressionPercent,
fileMaxViews,
},
headers: req.headers,
},
});
if (req.headers.zws) invis = await createInvisImage(zconfig.uploader.length, file.id);
await datasource.save(file.name, Buffer.from(chunks));
logger.info(`User ${user.username} (${user.id}) uploaded ${file.name} (${file.id}) (chunked)`);
let domain;
if (req.headers['override-domain']) {
domain = `${zconfig.core.return_https ? 'https' : 'http'}://${req.headers['override-domain']}`;
} else if (user.domains.length) {
domain = user.domains[Math.floor(Math.random() * user.domains.length)];
} else {
domain = `${zconfig.core.return_https ? 'https' : 'http'}://${req.headers.host}`;
}
const responseUrl = `${domain}${zconfig.uploader.route === '/' ? '/' : zconfig.uploader.route + '/'}${
invis ? invis.invis : encodeURI(file.name)
}`;
response.files.push(responseUrl);
if (zconfig.discord?.upload) {
await sendUpload(user, file, `${domain}/r/${invis ? invis.invis : file.name}`, responseUrl);
}
if (zconfig.exif.enabled && zconfig.exif.remove_gps && mimetype.startsWith('image/')) {
try {
await removeGPSData(file);
response.removed_gps = true;
} catch (e) {
logger.error(`Failed to remove GPS data from ${file.name} (${file.id}) - ${e.message}`);
response.removed_gps = false;
}
}
return res.json(response);
return res.json({
pending: true,
});
}
return res.json({
@@ -260,7 +200,7 @@ async function handler(req: NextApiReq, res: NextApiRes) {
let mimetype = file.mimetype;
if (file.mimetype === 'application/octet-stream' && zconfig.uploader.assume_mimetypes) {
const ext = file.originalname.split('.').pop();
const ext = parse(file.originalname).ext.replace('.', '');
const mime = await guess(ext);
if (!mime) response.assumed_mimetype = false;
@@ -315,7 +255,12 @@ async function handler(req: NextApiReq, res: NextApiRes) {
response.files.push(responseUrl);
if (zconfig.discord?.upload) {
await sendUpload(user, fileUpload, `${domain}/r/${invis ? invis.invis : fileUpload.name}`, responseUrl);
await sendUpload(
user,
fileUpload,
`${domain}/r/${invis ? invis.invis : encodeURI(fileUpload.name)}`,
responseUrl
);
}
if (zconfig.exif.enabled && zconfig.exif.remove_gps && fileUpload.mimetype.startsWith('image/')) {
+8
View File
@@ -14,6 +14,10 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
where: {
id: Number(id),
},
include: {
files: true,
Folder: true,
},
});
if (!target) return res.notFound('user not found');
@@ -175,6 +179,10 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
} else {
delete target.password;
if (user.superAdmin && target.superAdmin) delete target.files;
if (user.administrator && !user.superAdmin && (target.administrator || target.superAdmin))
delete target.files;
return res.json(target);
}
}
+4 -5
View File
@@ -5,7 +5,6 @@ import datasource from 'lib/datasource';
import Logger from 'lib/logger';
import prisma from 'lib/prisma';
import { NextApiReq, NextApiRes, UserExtended, withZipline } from 'middleware/withZipline';
import { tmpdir } from 'os';
import { join } from 'path';
const logger = Logger.get('user::export');
@@ -22,7 +21,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
const zip = new Zip();
const export_name = `zipline_export_${user.id}_${Date.now()}.zip`;
const path = join(tmpdir(), export_name);
const path = join(config.core.temp_directory, export_name);
logger.debug(`creating write stream at ${path}`);
const write_stream = createWriteStream(path);
@@ -121,18 +120,18 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
const parts = export_name.split('_');
if (Number(parts[2]) !== user.id) return res.unauthorized('cannot access export owned by another user');
const stream = createReadStream(join(tmpdir(), export_name));
const stream = createReadStream(join(config.core.temp_directory, export_name));
res.setHeader('Content-Type', 'application/zip');
res.setHeader('Content-Disposition', `attachment; filename="${export_name}"`);
stream.pipe(res);
} else {
const files = await readdir(tmpdir());
const files = await readdir(config.core.temp_directory);
const exp = files.filter((f) => f.startsWith('zipline_export_'));
const exports = [];
for (let i = 0; i !== exp.length; ++i) {
const name = exp[i];
const stats = await stat(join(tmpdir(), name));
const stats = await stat(join(config.core.temp_directory, name));
if (Number(exp[i].split('_')[2]) !== user.id) continue;
exports.push({ name, size: stats.size });
+62 -4
View File
@@ -31,15 +31,46 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
} else {
if (!req.body.id) return res.badRequest('no file id');
const file = await prisma.file.delete({
let file = await prisma.file.findFirst({
where: {
id: req.body.id,
userId: user.id,
},
include: {
user: {
select: {
administrator: true,
superAdmin: true,
username: true,
id: true,
},
},
},
});
if (!file && (!user.administrator || !user.superAdmin)) return res.notFound('file not found');
file = await prisma.file.delete({
where: {
id: req.body.id,
},
include: {
user: {
select: {
administrator: true,
superAdmin: true,
username: true,
id: true,
},
},
},
});
await datasource.delete(file.name);
logger.info(`User ${user.username} (${user.id}) deleted an image ${file.name} (${file.id})`);
logger.info(
`User ${user.username} (${user.id}) deleted an image ${file.name} (${file.id}) owned by ${file.user.username} (${file.user.id})`
);
// @ts-ignore
if (file.password) file.password = true;
@@ -51,14 +82,33 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
let file;
if (req.body.favorite !== null)
if (req.body.favorite !== null) {
file = await prisma.file.findFirst({
where: {
id: req.body.id,
userId: user.id,
},
include: {
user: {
select: {
administrator: true,
superAdmin: true,
username: true,
id: true,
},
},
},
});
if (!file && (!user.administrator || !user.superAdmin)) return res.notFound('file not found');
file = await prisma.file.update({
where: { id: req.body.id },
data: {
favorite: req.body.favorite,
},
});
}
// @ts-ignore
if (file.password) file.password = true;
return res.json(file);
@@ -83,6 +133,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
maxViews: number;
views: number;
size: number;
originalName: string;
thumbnail?: { name: string };
}[] = await prisma.file.findMany({
where: {
userId: user.id,
@@ -102,11 +154,17 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
folderId: true,
maxViews: true,
size: true,
originalName: true,
thumbnail: true,
},
});
for (let i = 0; i !== files.length; ++i) {
(files[i] as unknown as { url: string }).url = formatRootUrl(config.uploader.route, files[i].name);
if (files[i].thumbnail) {
(files[i].thumbnail as unknown as string) = formatRootUrl('/r', files[i].thumbnail.name);
}
}
if (req.query.filter && req.query.filter === 'media')
+3 -2
View File
@@ -58,12 +58,13 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json(folder);
} else {
if (req.query.files instanceof Array) req.query.files = req.query.files[0];
const folders = await prisma.folder.findMany({
where: {
userId: user.id,
},
select: {
files: !!req.query.files,
files: ((req.query.files as string) ?? 'false').toLowerCase() === 'true',
id: true,
name: true,
userId: true,
@@ -76,7 +77,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
},
});
if (req.query.files) {
if (((req.query.files as string) ?? 'false').toLowerCase() === 'true') {
for (let i = 0; i !== folders.length; ++i) {
const folder = folders[i];
for (let j = 0; j !== folders[i].files.length; ++j) {
+22 -14
View File
@@ -1,4 +1,4 @@
import config from 'lib/config';
import zconfig from 'lib/config';
import Logger from 'lib/logger';
import { discord_auth, github_auth, google_auth } from 'lib/oauth';
import prisma from 'lib/prisma';
@@ -18,7 +18,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json({
error: 'oauth token expired',
redirect_uri: github_auth.oauth_url(config.oauth.github_client_id),
redirect_uri: github_auth.oauth_url(zconfig.oauth.github_client_id),
});
}
} else if (user.oauth.find((o) => o.provider === 'DISCORD')) {
@@ -35,8 +35,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json({
error: 'oauth token expired',
redirect_uri: discord_auth.oauth_url(
config.oauth.discord_client_id,
`${config.core.return_https ? 'https' : 'http'}://${req.headers.host}`
zconfig.oauth.discord_client_id,
`${zconfig.core.return_https ? 'https' : 'http'}://${req.headers.host}`
),
});
}
@@ -47,8 +47,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: new URLSearchParams({
client_id: config.oauth.discord_client_id,
client_secret: config.oauth.discord_client_secret,
client_id: zconfig.oauth.discord_client_id,
client_secret: zconfig.oauth.discord_client_secret,
grant_type: 'refresh_token',
refresh_token: provider.refresh,
}),
@@ -59,8 +59,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json({
error: 'oauth token expired',
redirect_uri: discord_auth.oauth_url(
config.oauth.discord_client_id,
`${config.core.return_https ? 'https' : 'http'}://${req.headers.host}`
zconfig.oauth.discord_client_id,
`${zconfig.core.return_https ? 'https' : 'http'}://${req.headers.host}`
),
});
}
@@ -90,8 +90,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json({
error: 'oauth token expired',
redirect_uri: google_auth.oauth_url(
config.oauth.google_client_id,
`${config.core.return_https ? 'https' : 'http'}://${req.headers.host}`
zconfig.oauth.google_client_id,
`${zconfig.core.return_https ? 'https' : 'http'}://${req.headers.host}`
),
});
}
@@ -101,8 +101,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: new URLSearchParams({
client_id: config.oauth.google_client_id,
client_secret: config.oauth.google_client_secret,
client_id: zconfig.oauth.google_client_id,
client_secret: zconfig.oauth.google_client_secret,
grant_type: 'refresh_token',
refresh_token: provider.refresh,
}),
@@ -113,8 +113,8 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
return res.json({
error: 'oauth token expired',
redirect_uri: google_auth.oauth_url(
config.oauth.google_client_id,
`${config.core.return_https ? 'https' : 'http'}://${req.headers.host}`
zconfig.oauth.google_client_id,
`${zconfig.core.return_https ? 'https' : 'http'}://${req.headers.host}`
),
});
}
@@ -241,6 +241,14 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
}
}
export const config = {
api: {
bodyParser: {
sizeLimit: '50mb',
},
},
};
export default withZipline(handler, {
methods: ['GET', 'PATCH'],
user: true,
+33 -3
View File
@@ -1,3 +1,5 @@
import { Prisma } from '@prisma/client';
import { s } from '@sapphire/shapeshift';
import config from 'lib/config';
import prisma from 'lib/prisma';
import { formatRootUrl } from 'lib/utils/urls';
@@ -5,12 +7,27 @@ import { NextApiReq, NextApiRes, UserExtended, withZipline } from 'middleware/wi
const pageCount = 16;
const sortByValidator = s.enum(
...([
'createdAt',
'views',
'expiresAt',
'size',
'name',
'mimetype',
] satisfies (keyof Prisma.FileOrderByWithRelationInput)[])
);
const orderValidator = s.enum('asc', 'desc');
async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
const { page, filter, count, favorite } = req.query as {
const { page, filter, count, favorite, ...rest } = req.query as {
page: string;
filter: string;
count: string;
favorite: string;
sortBy: string;
order: string;
};
const where = {
@@ -33,7 +50,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
},
],
}),
};
} satisfies Prisma.FileWhereInput;
if (count) {
const count = await prisma.file.count({
@@ -48,6 +65,14 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
if (!page) return res.badRequest('no page');
if (isNaN(Number(page))) return res.badRequest('page is not a number');
// validate sortBy
const sortBy = sortByValidator.run(rest.sortBy || 'createdAt');
if (!sortBy.isOk()) return res.badRequest('invalid sortBy option');
// validate order
const order = orderValidator.run(rest.order || 'desc');
if (!sortBy.isOk()) return res.badRequest('invalid order option');
const files: {
favorite: boolean;
createdAt: Date;
@@ -60,10 +85,11 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
folderId: number;
size: number;
password: string | boolean;
thumbnail?: { name: string };
}[] = await prisma.file.findMany({
where,
orderBy: {
createdAt: 'desc',
[sortBy.value]: order.value,
},
select: {
createdAt: true,
@@ -77,6 +103,7 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
folderId: true,
size: true,
password: true,
thumbnail: true,
},
skip: page ? (Number(page) - 1) * pageCount : undefined,
take: page ? pageCount : undefined,
@@ -87,6 +114,9 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
if (file.password) file.password = true;
(file as unknown as { url: string }).url = formatRootUrl(config.uploader.route, file.name);
if (files[i].thumbnail) {
(files[i].thumbnail as unknown as string) = formatRootUrl('/r', files[i].thumbnail.name);
}
}
return res.json(files);
+40
View File
@@ -0,0 +1,40 @@
import prisma from 'lib/prisma';
import { NextApiReq, NextApiRes, UserExtended, withZipline } from 'middleware/withZipline';
async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
if (req.method === 'DELETE') {
const fileIds = req.body.id as number[];
const existingFiles = await prisma.incompleteFile.findMany({
where: {
id: {
in: fileIds,
},
userId: user.id,
},
});
const incFiles = await prisma.incompleteFile.deleteMany({
where: {
id: {
in: existingFiles.map((x) => x.id),
},
},
});
return res.json(incFiles);
} else {
const files = await prisma.incompleteFile.findMany({
where: {
userId: user.id,
},
});
return res.json(files);
}
}
export default withZipline(handler, {
methods: ['GET', 'DELETE'],
user: true,
});
+4
View File
@@ -27,11 +27,15 @@ async function handler(req: NextApiReq, res: NextApiRes, user: UserExtended) {
folderId: true,
size: true,
favorite: true,
thumbnail: true,
},
});
for (let i = 0; i !== files.length; ++i) {
(files[i] as unknown as { url: string }).url = formatRootUrl(config.uploader.route, files[i].name);
if (files[i].thumbnail) {
(files[i].thumbnail as unknown as string) = formatRootUrl('/r', files[i].thumbnail.name);
}
}
if (req.query.filter && req.query.filter === 'media')
+49 -33
View File
@@ -22,7 +22,13 @@ import { useRouter } from 'next/router';
import { useEffect, useState } from 'react';
export { getServerSideProps } from 'middleware/getServerSideProps';
export default function Login({ title, user_registration, oauth_registration, oauth_providers: unparsed }) {
export default function Login({
title,
user_registration,
oauth_registration,
bypass_local_login,
oauth_providers: unparsed,
}) {
const router = useRouter();
// totp modal
@@ -34,6 +40,9 @@ export default function Login({ title, user_registration, oauth_registration, oa
const oauth_providers = JSON.parse(unparsed);
const show_local_login =
router.query.local === 'true' || !(bypass_local_login && oauth_providers?.length > 0);
const icons = {
GitHub: IconBrandGithub,
Discord: IconBrandDiscordFilled,
@@ -99,6 +108,12 @@ export default function Login({ title, user_registration, oauth_registration, oa
useEffect(() => {
(async () => {
// if the user includes `local=true` as a query param, show the login form
// otherwise, redirect to the oauth login if there is only one registered provider
if (bypass_local_login && oauth_providers?.length === 1 && router.query.local !== 'true') {
await router.push(oauth_providers[0].url);
}
const a = await fetch('/api/user');
if (a.ok) await router.push('/dashboard');
})();
@@ -152,7 +167,7 @@ export default function Login({ title, user_registration, oauth_registration, oa
<Center sx={{ height: '100vh' }}>
<Card radius='md'>
<Title size={30} align='left'>
{title}
{bypass_local_login ? ` Login to ${title} with` : title}
</Title>
{oauth_registration && (
@@ -165,7 +180,7 @@ export default function Login({ title, user_registration, oauth_registration, oa
variant='outline'
radius='md'
fullWidth
leftIcon={<Icon height={'15'} width={'15'} />}
leftIcon={<Icon size='1rem' />}
my='xs'
component={Link}
href={url}
@@ -174,41 +189,42 @@ export default function Login({ title, user_registration, oauth_registration, oa
</Button>
))}
</Group>
<Divider my='xs' label='or' labelPosition='center' />
{show_local_login && <Divider my='xs' label='or' labelPosition='center' />}
</>
)}
<form onSubmit={form.onSubmit((v) => onSubmit(v))}>
<TextInput
my='xs'
radius='md'
size='md'
id='username'
label='Username'
{...form.getInputProps('username')}
/>
<PasswordInput
my='xs'
radius='md'
size='md'
id='password'
label='Password'
{...form.getInputProps('password')}
/>
{show_local_login && (
<form onSubmit={form.onSubmit((v) => onSubmit(v))}>
<TextInput
my='xs'
radius='md'
size='md'
id='username'
label='Username'
{...form.getInputProps('username')}
/>
<PasswordInput
my='xs'
radius='md'
size='md'
id='password'
label='Password'
{...form.getInputProps('password')}
/>
<Group position='apart'>
{user_registration && (
<Anchor size='xs' href='/auth/register' component={Link}>
Don&apos;t have an account? Register
</Anchor>
)}
<Group position='apart'>
{user_registration && (
<Anchor size='xs' href='/auth/register' component={Link}>
Don&apos;t have an account? Register
</Anchor>
)}
<Button size='sm' p='xs' radius='md' my='xs' type='submit' loading={loading}>
Login
</Button>
</Group>
</form>
<Button size='sm' p='xs' radius='md' my='xs' type='submit' loading={loading}>
Login
</Button>
</Group>
</form>
)}
</Card>
</Center>
</>
+42
View File
@@ -0,0 +1,42 @@
import { LoadingOverlay } from '@mantine/core';
import Layout from 'components/Layout';
import UserFiles from 'components/pages/Users/UserFiles';
import useLogin from 'hooks/useLogin';
import Head from 'next/head';
import { getServerSideProps as middlewareProps } from 'middleware/getServerSideProps';
import { GetServerSideProps } from 'next';
export default function UsersId(props) {
const { loading } = useLogin();
if (loading) return <LoadingOverlay visible={loading} />;
const title = `${props.title} - User - ${props.userId}`;
return (
<>
<Head>
<title>{title}</title>
</Head>
<Layout props={props}>
<UserFiles
userId={props.userId}
disableMediaPreview={props.disable_media_preview}
exifEnabled={props.exif_enabled}
compress={props.compress}
/>
</Layout>
</>
);
}
export const getServerSideProps: GetServerSideProps = async (context) => {
const { id } = context.params as { id: string };
// @ts-ignore
const { props } = await middlewareProps(context);
return {
props: {
userId: id,
...props,
},
};
};
@@ -10,7 +10,7 @@ export default function UsersPage(props) {
if (loading) return <LoadingOverlay visible={loading} />;
const title = `${props.title} - User`;
const title = `${props.title} - Users`;
return (
<>
<Head>
+3 -2
View File
@@ -12,7 +12,8 @@ export default function OauthError({ error, provider }) {
useEffect(() => {
const interval = setInterval(() => {
setRemaining((remaining) => remaining - 1);
if (remaining > 0) setRemaining((remaining) => remaining - 1);
else clearInterval(interval);
}, 1000);
return () => clearInterval(interval);
@@ -43,7 +44,7 @@ export default function OauthError({ error, provider }) {
</Title>
<MutedText sx={{ fontSize: 40, fontWeight: 500 }}>{error}</MutedText>
<MutedText>
Redirecting to login in {remaining} second{remaining === 1 ? 's' : ''}
Redirecting to login in {remaining} second{remaining !== 1 ? 's' : ''}
</MutedText>
<Button component={Link} href='/dashboard'>
Head to the Dashboard
+2 -7
View File
@@ -1,7 +1,6 @@
import { Box, Button, Modal, PasswordInput } from '@mantine/core';
import type { File } from '@prisma/client';
import AnchorNext from 'components/AnchorNext';
import config from 'lib/config';
import exts from 'lib/exts';
import prisma from 'lib/prisma';
import { parseString } from 'lib/utils/parser';
@@ -17,18 +16,15 @@ export default function EmbeddedFile({
user,
pass,
prismRender,
onDash,
compress,
}: {
file: File & { imageProps?: HTMLImageElement };
user: UserExtended;
pass: boolean;
prismRender: boolean;
onDash: boolean;
compress?: boolean;
}) {
const dataURL = (route: string) =>
`${route}/${encodeURI(file.name)}?compress=${compress == null ? onDash : compress}`;
const dataURL = (route: string) => `${route}/${encodeURI(file.name)}?compress=${compress ?? false}`;
const router = useRouter();
const [opened, setOpened] = useState(pass);
@@ -235,7 +231,7 @@ export const getServerSideProps: GetServerSideProps = async (context) => {
if (file.password) file.password = true;
return {
props: {
image: file,
file,
user,
pass,
prismRender: true,
@@ -268,7 +264,6 @@ export const getServerSideProps: GetServerSideProps = async (context) => {
file,
user,
pass: file.password ? true : false,
onDash: config.core.compression.on_dashboard,
compress,
},
};
+28
View File
@@ -0,0 +1,28 @@
import config from 'lib/config';
import { readdir, rm } from 'fs/promises';
import { join } from 'path';
import { existsSync } from 'fs';
async function main() {
const temp = config.core.temp_directory;
if (!existsSync(temp)) {
console.log('Temp directory does not exist, exiting..');
process.exit(0);
}
const files = (await readdir(temp)).filter((x) => x.startsWith('zipline_partial_'));
if (files.length === 0) {
console.log('No partial files found, exiting..');
process.exit(0);
} else {
for (const file of files) {
console.log(`Deleting ${file}`);
await rm(join(temp, file));
}
console.log('Done!');
process.exit(0);
}
}
main();
+35 -2
View File
@@ -8,13 +8,42 @@ async function main() {
await migrations();
const prisma = new PrismaClient();
let notFound = false;
const files = await prisma.file.findMany();
const files = await prisma.file.findMany({
...(process.argv.includes('--force-update')
? undefined
: {
where: {
size: 0,
},
}),
select: {
id: true,
name: true,
size: true,
},
});
console.log(`The script will attempt to query the size of ${files.length} files.`);
for (let i = 0; i !== files.length; ++i) {
const file = files[i];
if (!(await datasource.get(file.name))) {
if (process.argv.includes('--force-delete')) {
console.log(`File ${file.name} does not exist. Deleting...`);
await prisma.file.delete({
where: {
id: file.id,
},
});
continue;
} else {
notFound ? null : (notFound = true);
continue;
}
}
const size = await datasource.size(file.name);
if (size === 0) {
console.log(`File ${file.name} has a size of 0 bytes. Ignoring...`);
@@ -31,7 +60,11 @@ async function main() {
}
}
console.log('Done.');
notFound
? console.log(
'At least one file has been found to not exist in the datasource but was on the database. To remove these files, run the script with the --force-delete flag.'
)
: console.log('Done.');
process.exit(0);
}
+2 -1
View File
@@ -1,3 +1,4 @@
import config from 'lib/config';
import { inspect } from 'util';
console.log(JSON.stringify(config, null, 2));
console.log(inspect(config, { depth: Infinity, colors: true }));
+2
View File
@@ -7,6 +7,8 @@ function postUrlDecorator(fastify: FastifyInstance, _, done) {
done();
async function postUrl(this: FastifyReply, url: Url) {
if (!url) return true;
const nUrl = await this.server.prisma.url.update({
where: {
id: url.id,
+1
View File
@@ -7,6 +7,7 @@ function preFileDecorator(fastify: FastifyInstance, _, done) {
done();
async function preFile(this: FastifyReply, file: File) {
if (file.favorite) return false;
if (file.expiresAt && file.expiresAt < new Date()) {
await this.server.datasource.delete(file.name);
await this.server.prisma.file.delete({ where: { id: file.id } });
+36
View File
@@ -21,6 +21,7 @@ import prismaPlugin from './plugins/prisma';
import rawRoute from './routes/raw';
import uploadsRoute, { uploadsRouteOnResponse } from './routes/uploads';
import urlsRoute, { urlsRouteOnResponse } from './routes/urls';
import { Worker } from 'worker_threads';
const dev = process.env.NODE_ENV === 'development';
const logger = Logger.get('server');
@@ -100,6 +101,18 @@ async function start() {
return reply.type('image/x-icon').send(favicon);
});
if (config.features.robots_txt) {
server.get('/robots.txt', async (_, reply) => {
return reply.type('text/plain').send(`User-Agent: *
Disallow: /r/
Disallow: /api/
Disallow: /view/
Disallow: ${config.uploader.route}
Disallow: ${config.urls.route}
`);
});
}
// makes sure to handle both in one route as you cant have two handlers with the same route
if (config.urls.route === '/' && config.uploader.route === '/') {
server.route({
@@ -171,9 +184,11 @@ async function start() {
await clearInvites.bind(server)();
await stats.bind(server)();
await thumbs.bind(server)();
setInterval(() => clearInvites.bind(server)(), config.core.invites_interval * 1000);
setInterval(() => stats.bind(server)(), config.core.stats_interval * 1000);
setInterval(() => thumbs.bind(server)(), config.core.thumbnails_interval * 1000);
}
async function stats(this: FastifyInstance) {
@@ -205,6 +220,27 @@ async function clearInvites(this: FastifyInstance) {
logger.child('invites').debug(`deleted ${count} used invites`);
}
async function thumbs(this: FastifyInstance) {
const videoFiles = await this.prisma.file.findMany({
where: {
mimetype: {
startsWith: 'video/',
},
thumbnail: null,
},
});
logger.child('thumb').debug(`found ${videoFiles.length} videos without thumbnails`);
for (const file of videoFiles) {
new Worker('./dist/worker/thumbnail.js', {
workerData: {
id: file.id,
},
});
}
}
function genFastifyOpts(): FastifyServerOptions {
const opts = {};
+23 -2
View File
@@ -1,6 +1,7 @@
import { FastifyInstance } from 'fastify';
import fastifyPlugin from 'fastify-plugin';
import { mkdir } from 'fs/promises';
import { existsSync } from 'fs';
import { mkdir, readdir } from 'fs/promises';
import type { Config } from 'lib/config/Config';
async function configPlugin(fastify: FastifyInstance, config: Config) {
@@ -16,7 +17,9 @@ async function configPlugin(fastify: FastifyInstance, config: Config) {
.error(
'The config file is located at `.env.local`, or if using docker-compose you can change the variables in the `docker-compose.yml` file.'
)
.error('It is recomended to use a secret that is alphanumeric and randomized.')
.error(
'It is recomended to use a secret that is alphanumeric and randomized. If you include special characters, surround the secret with quotes.'
)
.error('A way you can generate this is through a password manager you may have.');
process.exit(1);
@@ -26,6 +29,24 @@ async function configPlugin(fastify: FastifyInstance, config: Config) {
await mkdir(config.datasource.local.directory, { recursive: true });
}
if (!existsSync(config.core.temp_directory)) {
await mkdir(config.core.temp_directory, { recursive: true });
} else {
const files = await readdir(config.core.temp_directory);
if (
files.filter((x: string) => x.startsWith('zipline_partial_') || x.startsWith('zipline-exif-read-'))
.length > 0
)
fastify.logger
.error("Found temporary files in Zipline's temp directory.")
.error('This can happen if Zipline crashes or is stopped while chunking a file.')
.error(
'If you are sure that no files are currently being processed, you can delete the files in the temp directory.'
)
.error('The temp directory is located at: ' + config.core.temp_directory)
.error('If you are unsure, you can safely ignore this message.');
}
return;
}
+1
View File
@@ -61,6 +61,7 @@ export async function migrations() {
logger.error(
`Unable to connect to database \`${process.env.DATABASE_URL}\`, check your database connection`
);
logger.debug(error);
} else {
logger.error('Failed to migrate database... exiting...');
logger.error(error);
+108
View File
@@ -0,0 +1,108 @@
import { File } from '@prisma/client';
import { spawn } from 'child_process';
import ffmpeg from 'ffmpeg-static';
import { createWriteStream } from 'fs';
import { rm } from 'fs/promises';
import config from 'lib/config';
import datasource from 'lib/datasource';
import Logger from 'lib/logger';
import prisma from 'lib/prisma';
import { join } from 'path';
import { isMainThread, workerData } from 'worker_threads';
const { id } = workerData as { id: number };
const logger = Logger.get('worker::thumbnail').child(id.toString() ?? 'unknown-ident');
if (isMainThread) {
logger.error('worker is not a thread');
process.exit(1);
}
async function loadThumbnail(path) {
const args = ['-i', path, '-frames:v', '1', '-f', 'mjpeg', 'pipe:1'];
const child = spawn(ffmpeg, args, { stdio: ['ignore', 'pipe', 'ignore'] });
const data: Promise<Buffer> = new Promise((resolve, reject) => {
child.stdout.once('data', resolve);
child.once('error', reject);
});
return data;
}
async function loadFileTmp(file: File) {
const stream = await datasource.get(file.name);
// pipe to tmp file
const tmpFile = join(config.core.temp_directory, `zipline_thumb_${file.id}_${file.id}.tmp`);
const fileWriteStream = createWriteStream(tmpFile);
await new Promise((resolve, reject) => {
stream.pipe(fileWriteStream);
stream.once('error', reject);
stream.once('end', resolve);
});
return tmpFile;
}
async function start() {
const file = await prisma.file.findUnique({
where: {
id,
},
include: {
thumbnail: true,
},
});
if (!file) {
logger.error('file not found');
process.exit(1);
}
if (!file.mimetype.startsWith('video/')) {
logger.info('file is not a video');
process.exit(0);
}
if (file.thumbnail) {
logger.info('thumbnail already exists');
process.exit(0);
}
const tmpFile = await loadFileTmp(file);
logger.debug(`loaded file to tmp: ${tmpFile}`);
const thumbnail = await loadThumbnail(tmpFile);
logger.debug(`loaded thumbnail: ${thumbnail.length} bytes mjpeg`);
const { thumbnail: thumb } = await prisma.file.update({
where: {
id: file.id,
},
data: {
thumbnail: {
create: {
name: `.thumb-${file.id}.jpg`,
},
},
},
select: {
thumbnail: true,
},
});
await datasource.save(thumb.name, thumbnail);
logger.info(`thumbnail saved - ${thumb.name}`);
logger.debug(`thumbnail ${JSON.stringify(thumb)}`);
logger.debug(`removing tmp file: ${tmpFile}`);
await rm(tmpFile);
process.exit(0);
}
start();
+224
View File
@@ -0,0 +1,224 @@
import { readdir, readFile, open, rm } from 'fs/promises';
import type { NameFormat } from 'lib/format';
import Logger from 'lib/logger';
import type { UserExtended } from 'middleware/withZipline';
import { isMainThread, workerData } from 'worker_threads';
import prisma from 'lib/prisma';
import { join } from 'path';
import { IncompleteFile, InvisibleFile } from '@prisma/client';
import { removeGPSData } from 'lib/utils/exif';
import { sendUpload } from 'lib/discord';
import { createInvisImage, hashPassword } from 'lib/util';
import formatFileName from 'lib/format';
export type UploadWorkerData = {
user: UserExtended;
file: {
filename: string;
mimetype: string;
identifier: string;
lastchunk: boolean;
totalBytes: number;
};
response: {
expiresAt?: Date;
format: NameFormat;
imageCompressionPercent?: number;
fileMaxViews?: number;
};
headers: Record<string, string>;
};
const { user, file, response, headers } = workerData as UploadWorkerData;
const logger = Logger.get('worker::upload').child(file?.identifier ?? 'unknown-ident');
if (isMainThread) {
logger.error('worker is not a thread');
process.exit(1);
}
if (!file.lastchunk) {
logger.error('lastchunk is false, worker should not have been started');
process.exit(1);
}
if (!config.chunks.enabled) {
logger.error('chunks are not enabled, worker should not have been started');
process.exit(1);
}
start();
async function start() {
logger.debug('starting worker');
const partials = await readdir(config.core.temp_directory).then((files) =>
files.filter((x) => x.startsWith(`zipline_partial_${file.identifier}`))
);
const readChunks = partials.map((x) => {
const [, , , start, end] = x.split('_');
return { start: Number(start), end: Number(end), filename: x };
});
const incompleteFile = await prisma.incompleteFile.create({
data: {
data: {
file,
},
chunks: readChunks.length,
chunksComplete: 0,
status: 'PENDING',
userId: user.id,
},
});
const compressionUsed = response.imageCompressionPercent && file.mimetype.startsWith('image/');
const ext = file.filename.split('.').length === 1 ? '' : file.filename.split('.').pop();
const fileName = await formatFileName(response.format, file.filename);
let fd;
if (config.datasource.type === 'local') {
fd = await open(
join(
config.datasource.local.directory,
`${fileName}${compressionUsed ? '.jpg' : `${ext ? '.' : ''}${ext}`}`
),
'w'
);
} else {
fd = new Uint8Array(file.totalBytes);
}
for (let i = 0; i !== readChunks.length; ++i) {
const chunk = readChunks[i];
const buffer = await readFile(join(config.core.temp_directory, chunk.filename));
if (config.datasource.type === 'local') {
const { bytesWritten } = await fd.write(buffer, 0, buffer.length, chunk.start);
logger.child('fd').debug(`wrote ${bytesWritten} bytes to file`);
} else {
fd.set(buffer, chunk.start);
logger.child('bytes').debug(`wrote ${buffer.length} bytes to array`);
}
await rm(join(config.core.temp_directory, chunk.filename));
await prisma.incompleteFile.update({
where: {
id: incompleteFile.id,
},
data: {
chunksComplete: {
increment: 1,
},
status: 'PROCESSING',
},
});
}
if (config.datasource.type === 'local') {
await fd.close();
} else {
logger.debug('writing file to datasource');
await datasource.save(
`${fileName}${compressionUsed ? '.jpg' : `${ext ? '.' : ''}${ext}`}`,
Buffer.from(fd as Uint8Array)
);
}
const final = await prisma.incompleteFile.update({
where: {
id: incompleteFile.id,
},
data: {
status: 'COMPLETE',
},
});
logger.debug('done writing file');
await runFileComplete(fileName, ext, compressionUsed, final);
logger.debug('done running worker');
process.exit(0);
}
async function setResponse(incompleteFile: IncompleteFile, code: number, message: string) {
incompleteFile.data['code'] = code;
incompleteFile.data['message'] = message;
return prisma.incompleteFile.update({
where: {
id: incompleteFile.id,
},
data: {
data: incompleteFile.data,
},
});
}
async function runFileComplete(
fileName: string,
ext: string,
compressionUsed: boolean,
incompleteFile: IncompleteFile
) {
if (config.uploader.disabled_extensions.includes(ext))
return setResponse(incompleteFile, 403, 'disabled extension');
let password = null;
if (headers.password) {
password = await hashPassword(headers.password as string);
}
let invis: InvisibleFile;
const fFile = await prisma.file.create({
data: {
name: `${fileName}${compressionUsed ? '.jpg' : `${ext ? '.' : ''}${ext}`}`,
mimetype: file.mimetype,
userId: user.id,
embed: !!headers.embed,
password,
expiresAt: response.expiresAt,
maxViews: response.fileMaxViews,
originalName: headers['original-name'] ? file.filename ?? null : null,
size: file.totalBytes,
},
});
if (headers.zws) invis = await createInvisImage(config.uploader.length, fFile.id);
logger.info(`User ${user.username} (${user.id}) uploaded ${fFile.name} (${fFile.id}) (chunked)`);
let domain;
if (headers['override-domain']) {
domain = `${config.core.return_https ? 'https' : 'http'}://${headers['override-domain']}`;
} else if (user.domains.length) {
domain = user.domains[Math.floor(Math.random() * user.domains.length)];
} else {
domain = `${config.core.return_https ? 'https' : 'http'}://${headers.host}`;
}
const responseUrl = `${domain}${config.uploader.route === '/' ? '/' : config.uploader.route + '/'}${
invis ? invis.invis : encodeURI(fFile.name)
}`;
if (config.discord?.upload) {
await sendUpload(user, fFile, `${domain}/r/${invis ? invis.invis : fFile.name}`, responseUrl);
}
if (config.exif.enabled && config.exif.remove_gps && fFile.mimetype.startsWith('image/')) {
try {
await removeGPSData(fFile);
} catch (e) {
logger.error(`Failed to remove GPS data from ${fFile.name} (${fFile.id}) - ${e.message}`);
}
}
await setResponse(incompleteFile, 200, responseUrl);
}
+16
View File
@@ -13,6 +13,17 @@ export default defineConfig([
entryPoints: ['src/server/index.ts'],
...opts,
},
// workers
{
entryPoints: ['src/worker/upload.ts'],
outDir: 'dist/worker',
...opts,
},
{
entryPoints: ['src/worker/thumbnail.ts'],
outDir: 'dist/worker',
...opts,
},
// scripts
{
entryPoints: ['src/scripts/import-dir.ts'],
@@ -44,4 +55,9 @@ export default defineConfig([
outDir: 'dist/scripts',
...opts,
},
{
entryPoints: ['src/scripts/clear-temp.ts'],
outDir: 'dist/scripts',
...opts,
},
]);
+79 -1
View File
@@ -1142,6 +1142,18 @@ __metadata:
languageName: node
linkType: hard
"@derhuerst/http-basic@npm:^8.2.0":
version: 8.2.4
resolution: "@derhuerst/http-basic@npm:8.2.4"
dependencies:
caseless: ^0.12.0
concat-stream: ^2.0.0
http-response-object: ^3.0.1
parse-cache-control: ^1.0.1
checksum: dfb2f30c23fb907988d1c34318fa74c54dcd3c3ba6b4b0e64cdb584d03303ad212dd3b3874328a9367d7282a232976acbd33a20bb9c7a6ea20752e879459253b
languageName: node
linkType: hard
"@emotion/babel-plugin@npm:^11.10.6":
version: 11.10.6
resolution: "@emotion/babel-plugin@npm:11.10.6"
@@ -2719,6 +2731,13 @@ __metadata:
languageName: node
linkType: hard
"@types/node@npm:^10.0.3":
version: 10.17.60
resolution: "@types/node@npm:10.17.60"
checksum: 2cdb3a77d071ba8513e5e8306fa64bf50e3c3302390feeaeff1fd325dd25c8441369715dfc8e3701011a72fed5958c7dfa94eb9239a81b3c286caa4d97db6eef
languageName: node
linkType: hard
"@types/node@npm:^17.0.45":
version: 17.0.45
resolution: "@types/node@npm:17.0.45"
@@ -3828,6 +3847,13 @@ __metadata:
languageName: node
linkType: hard
"caseless@npm:^0.12.0":
version: 0.12.0
resolution: "caseless@npm:0.12.0"
checksum: b43bd4c440aa1e8ee6baefee8063b4850fd0d7b378f6aabc796c9ec8cb26d27fb30b46885350777d9bd079c5256c0e1329ad0dc7c2817e0bb466810ebb353751
languageName: node
linkType: hard
"ccount@npm:^2.0.0":
version: 2.0.1
resolution: "ccount@npm:2.0.1"
@@ -4136,6 +4162,18 @@ __metadata:
languageName: node
linkType: hard
"concat-stream@npm:^2.0.0":
version: 2.0.0
resolution: "concat-stream@npm:2.0.0"
dependencies:
buffer-from: ^1.0.0
inherits: ^2.0.3
readable-stream: ^3.0.2
typedarray: ^0.0.6
checksum: d7f75d48f0ecd356c1545d87e22f57b488172811b1181d96021c7c4b14ab8855f5313280263dca44bb06e5222f274d047da3e290a38841ef87b59719bde967c7
languageName: node
linkType: hard
"console-control-strings@npm:^1.0.0, console-control-strings@npm:^1.1.0":
version: 1.1.0
resolution: "console-control-strings@npm:1.1.0"
@@ -5625,6 +5663,18 @@ __metadata:
languageName: node
linkType: hard
"ffmpeg-static@npm:^5.1.0":
version: 5.1.0
resolution: "ffmpeg-static@npm:5.1.0"
dependencies:
"@derhuerst/http-basic": ^8.2.0
env-paths: ^2.2.0
https-proxy-agent: ^5.0.0
progress: ^2.0.3
checksum: 0e27d671a0be1f585ef03e48c2af7c2be14f4e61470ffa02e3b8919551243ee854028a898dfcd16cdf1e3c01916f3c5e9938f42cbc7e877d7dd80d566867db8b
languageName: node
linkType: hard
"file-entry-cache@npm:^6.0.1":
version: 6.0.1
resolution: "file-entry-cache@npm:6.0.1"
@@ -6335,6 +6385,15 @@ __metadata:
languageName: node
linkType: hard
"http-response-object@npm:^3.0.1":
version: 3.0.2
resolution: "http-response-object@npm:3.0.2"
dependencies:
"@types/node": ^10.0.3
checksum: 6cbdcb4ce7b27c9158a131b772c903ed54add2ba831e29cc165e91c3969fa6f8105ddf924aac5b954b534ad15a1ae697b693331b2be5281ee24d79aae20c3264
languageName: node
linkType: hard
"https-proxy-agent@npm:5.0.1, https-proxy-agent@npm:^5.0.0":
version: 5.0.1
resolution: "https-proxy-agent@npm:5.0.1"
@@ -8874,6 +8933,13 @@ __metadata:
languageName: node
linkType: hard
"parse-cache-control@npm:^1.0.1":
version: 1.0.1
resolution: "parse-cache-control@npm:1.0.1"
checksum: 5a70868792124eb07c2dd07a78fcb824102e972e908254e9e59ce59a4796c51705ff28196d2b20d3b7353d14e9f98e65ed0e4eda9be072cc99b5297dc0466fee
languageName: node
linkType: hard
"parse-json@npm:^4.0.0":
version: 4.0.0
resolution: "parse-json@npm:4.0.0"
@@ -9301,7 +9367,7 @@ __metadata:
languageName: node
linkType: hard
"progress@npm:2.0.3":
"progress@npm:2.0.3, progress@npm:^2.0.3":
version: 2.0.3
resolution: "progress@npm:2.0.3"
checksum: f67403fe7b34912148d9252cb7481266a354bd99ce82c835f79070643bb3c6583d10dbcfda4d41e04bbc1d8437e9af0fb1e1f2135727878f5308682a579429b7
@@ -9736,6 +9802,17 @@ __metadata:
languageName: node
linkType: hard
"readable-stream@npm:^3.0.2":
version: 3.6.2
resolution: "readable-stream@npm:3.6.2"
dependencies:
inherits: ^2.0.3
string_decoder: ^1.1.1
util-deprecate: ^1.0.1
checksum: bdcbe6c22e846b6af075e32cf8f4751c2576238c5043169a1c221c92ee2878458a816a4ea33f4c67623c0b6827c8a400409bfb3cf0bf3381392d0b1dfb52ac8d
languageName: node
linkType: hard
"readable-stream@npm:^4.0.0":
version: 4.2.0
resolution: "readable-stream@npm:4.2.0"
@@ -11902,6 +11979,7 @@ __metadata:
fastify: ^4.15.0
fastify-plugin: ^4.5.0
fflate: ^0.7.4
ffmpeg-static: ^5.1.0
find-my-way: ^7.6.0
katex: ^0.16.4
mantine-datatable: ^2.2.6