02/14 - Thursday

How many times did you check if the forum was back up yesterday?

  • Total voters
    55
  • Poll closed .
Status
Not open for further replies.
Daniel

Daniel

Well-Known Member
Contributor
Joined
May 25, 2017
Messages
3,014
Reaction score
5,271
Points
1,338
Gender
Male
Title: Academic Survey: Antibiotic Use and Public Health | Accept
Requester: Dr. Rachel Smith [A2ULRPBBVY89DF] Contact
TV: [Hrly=$24.63] [Pay=Generous] [Approval=~24 hrs] [Comm=Unrated] [Rej=0] [Blk=0]
TO: [Pay=4.68] [Fast=5.00] [Comm=5.00] [Fair=5.00] [Reviews=31] [ToS=0]
TO2: [Hrly=19.75] [Pen=0.04] [Res=null] [Rec=100] [Rej=0] [ToS=0] [Brk=0]
Reward: 4.00
Duration: 2:00:00
Available: 1
Description: ACADEMIC RESEARCH: public opinion about antibiotic use and public health. After answering baseline questions, you will ask for your beliefs, opinions, and reactions to antibiotic resistance and strategies to slow resistance. 30-40 minutes
Qualifications: Total approved HITs GreaterThanOrEqualTo 100; DSIT2 taken NotEqualTo 1; HIT approval rate (%) GreaterThanOrEqualTo 95; Location EqualTo US
[tr][td]
HIT exported from Mturk Suite v2.6.4​
[/td][/tr]
[tr][td][/td][/tr]
 
  • Like
Reactions: Neemo and jklmnop
C to the J

C to the J

Outlaw Turker
Contributor
Master Pleaser
Crowd Pleaser
HIT Poster
Joined
Jan 12, 2016
Messages
53,232
Reaction score
163,959
Points
3,238
Age
37
Gender
Male
IridescentOwl

IridescentOwl

Active Member
Joined
Feb 2, 2019
Messages
568
Reaction score
1,247
Points
343
Gender
Female
What is this Apex you speak of and how do I play?
 
Metallica

Metallica

Never free, never me
Contributor
Joined
Jan 17, 2016
Messages
5,319
Reaction score
14,228
Points
1,113
Age
34
Gender
Male
Around 1 am yesterday morning, it was reported to me that the forum was inaccessible and was showing a database error landing page. A certificate expired, along with a database memory issue, yet I did not have proper access to rectify it. When I was put onto the DigitalOcean account, I had access to only one server and obviously this became an issue yesterday. We had to touch base with the "old" server guy to get this fixed. He ended up setting everything up as a managed cluster. He had to balance this with his job, which is why it took so long. Regardless, this makes administration of everything a fair bit easier in the long run. I now have access to everything like I should. It goes without saying I am terribly sorry all this happened, but now we should be a bit better off when this happens again. More detailed info below:

We are now using Kubernetes for the orchestration engine - the thing that manages how all the containers work together.

It uses TLS certificates to authenticate different pieces of the puzzle to talk to each other.

The master cert expired and there isn't an easy way (back then in that version of kubernetes) to renew those certs. It's now a fixed thing. Anyway, that expired certificate made the control utility (kubectl) useless so we couldn't restart the failed mysql db container pod.

So, the cluster cert had to be fixed before mysql could be fixed.

This basically meant redoing the kubernetes cluster.

It was decided to use the new managed kubernetes cluster option instead of rolling it again as it was done before to make it easier to admin going forward.

You won't have to worry about expiring certs and such. It's something DO takes care of.

However, this meant there was a need to copy various data from one place to another as well as export and reimport the mysql database as well as recreate the nginx, mysql, php, and memcached deployments.
 
Last edited:
  • Like
Reactions: SherryNotShirley, TotalBabe, jklmnop and 22 others
Status
Not open for further replies.
Top Bottom