𞋴𝛂𝛋𝛆

  • 165 Posts
  • 1.58K Comments
Joined 3 years ago
Cake day: June 9th, 2023
  • 𞋴𝛂𝛋𝛆@lemmy.worldtoOpen Source@lemmy.ml•a foos GenAi tool for daily usageEnglish
    3·
    3 days ago

    llama.cpp is at the core of almost all offline, open weights models. The server it creates is Open AI API compatible. Oobabooga Textgen WebUI is more user GUI oriented but based on llama.cpp. Oobabooga has the setup for loading models with a split workload between the CPU and GPU which makes larger gguf quantized models possible to run. Llama.cpp, has this feature, Oobabooga implements it. The model loading settings and softmax sampling settings take some trial and error to dial in well. It helps if you have a way of monitoring GPU memory usage in real time. Like I use a script that appends my terminal window title bar with GPU memory usage until inference time.

    Ollama is another common project people use for offline open weights models, and it also runs on top of llama.cpp. It is a lot easier to get started in some instances and several projects use Ollama as a baseline for “Hello World!” type stuff. It has pretty good model loading and softmax settings without any fuss, but it does this at the expense of only running on GPU or CPU but never both in a split workload. This may seem great at first, but if you never experience running much larger quantized models in the 30B-140B range, you are unlikely to have success or a positive experience overall. The much smaller models in the 4B-14B range are all that are likely to run fast enough on your hardware AND completely load in your GPU memory if you only have 8GB-24GB. Most of the newer models are actually Mixture of Experts architectures. This means it is like loading ~7 models initially, but then only inferencing two of them at any one time. All you need is the system memory or the Deepspeed package (uses disk drive for excess space required) to load these larger models. Larger quantized models are much much smarter and more capable. You also need llama.cpp if you want to use function calling for agentic behaviors. Look into the agentic API and pull history in this area of llama.cpp before selecting what models to test in depth.

    Huggingface is the goto website for sharing and sourcing models. That is heavily integrated with GitHub, so it is probably as toxic long term, but I do not know of a real FOSS alternative for that one. Hosting models is massive I/O for a server.

  • 𞋴𝛂𝛋𝛆@lemmy.worldtoNo Stupid Questions@lemmy.world•Starlink Alternative that can't be blockedEnglish
    25·
    7 days ago

    No. The primary way of blocking radio is by raising the noise floor across the band. The type of radio is irrelevant. The protocol is irrelevant. It is all only the electromagnetic spectrum from infrared light, to visible spectrum light, to radio light, through to xray or gamma ray light. How we divide that up into protocols, bands, and names is totally irrelevant. When transmitting radio light, we are all restricted in how much power we are allowed to send. All receiver circuits are listening for meaningful information above the noise floor. Bands are allocated to try to create spaces for certain types of communications. This controls the noise floor. Then electrical engineers design the hardware you buy to operate within this specification. If that noise floor is raised, the physical hardware is unable to retrieve information and effectively makes it useless. If you are a radio wizard and build your own transmitter that has more power, you just created a giant beacon that anyone will track easily to your location. Transmitting always reveals your exact location. In military operations, you constantly hear about some entity going radio silent. This is why. If you are a soldier, you may not carry a cell phone at all when on the job because it is constantly revealing your location. The only way to avoid this is with actually hard wire connections. You are able to use lasers for line of sight communications, but in practice, you will be limited by the optical lens focusing complexity and atmospheric distortion even from the ground with point to point regional communication. If anyone crosses the beam it will still be detected and is likely to leak some light depending on conditions and design.

    Ultimately, your only real option is the sneaker net which is damn near useless in US suburbia hell. Don’t forget that the freeway system was not created for the citizenry. It is only about military mobility. That is why the Germans made the autobahn and why the USA and others had to copy the idea. Your only defense is in the democratic political space.

  • 𞋴𝛂𝛋𝛆@lemmy.worldOPtopics@lemmy.world•Crew 11English
    2·
    7 days ago

    That is one of the original Ole Hanson homes. That far row are the ones on the sea cliff. Sucks because the train is just below them, and the cliffs are sedimentary marine deposition that has been uplifted on geological time scales. When it rains here in SoCal, and you hear about houses falling in landslides, this is one of those areas. None of these places are single family homes. That Ole Hanson one has six units IIRC. Only the ones on the cliff are super expensive.

  • 𞋴𝛂𝛋𝛆@lemmy.worldtoPrivacy@lemmy.ml•How do I check my router for malware?English
    3·
    2 months ago
    The easiest way I know of to check any machine is to put another router or machine in front of it with a white list firewall or way of logging DNS traffic. You just need to spot the address in the list.

    DNS filtering usually only filters on incoming packets, but for bot stuff that should catch issues.

    In general, most routers run everything from a serial flash chip on the board. These are usually 8, 16, or 32 megabytes. They have a simple bootloader like U-Boot. This is what loads the operating system. These devices have a UART serial port on the PCB. You can use a USB to serial UART adaptor to see what is happening in the device. With a proprietary OS, you are still likely to see the pre-init boot sequence that the bootloader prints to terminal. Most operating systems also print information to this interface, at least of the couple dozen junk devices I have been given and messed around with. I make a little mount for a USB to serial adaptor and add it to all of my routers when new, so I only need to plug in USB to get to the internal bootloader and tty terminal interface of OpenWRT. You will need to know the default baud rate of the device, although it is probably listed somewhere online or can be guessed as one of the common high values at or above 9600.

    Getting into this further gets complicated. It is probably better to look for any CVE that is relevant to the device or software and work backwards. Look for any software updates that have obfuscated the risk for each CVE. If the issue was not fixed, that is where to look to see if someone has exploited the device. Ultimately, they need clock cycles from the CPU scheduler. So it must be a process or some way of executing code from unregistered memory.

    This is getting to the edge of what I have messed around with and understand. There may be a way to get a memory map that includes unused pages, and compare that with a hex dump of the flash memory. This is outside of your scope of a proprietary OS, but hopefully frames the abstract scope of what is possible on this class of device when you have an open source stack. The main advantage of this kind of device and issue is that you can physically remove the flash chip and then see and manipulate every page and memory location. The device likely doesn’t have microcode loaded into the CPU(s) that make it challenging to determine what is going on.

    There is probably an easier way, but a hex dump of the current system can be hashed against the factory updated version to see if any differences are present. It is likely that any exploit will include a string with the address to connect to somewhere in flash memory. It could be obfuscated through encryption or a cypher, but a simple check for strings in the hex dump and a grep for “http” is a simple way to looks for issues.

    The OpenWRT forum is a good general source. The people behind the bootloaders for these devices are also Linux kernel developers and on the OpenWRT forum.

  • 𞋴𝛂𝛋𝛆@lemmy.worldtoShowerthoughts@lemmy.world•The family name goes first in East Asian names, reflecting the fact that they value family over individual; Whatever the parents' idea of "family" is, triumphs over the children's individuality.English
    84·
    2 months ago

    How do you punch holes in that dogma? I can think if many logical ways, but that is meaningless against the tribal structure.

    • If family is so valuable, why didn’t strong families usher in the present age of technology
    • intelligence, business acumen, and competency are not hereditary.
    • team sports are a capitalist marketing scam. Putting a blue jersey on your sperm does not make it relevant or better than purple jersey’d sperm.
    • patriarchal male culture is chauvinistic ineptitude and masochism marketed as a replacement for intelligence. It is an admission of subservience to those that dominate by thought and fundamental logic. Fools only fear a brute, civilizations fear a physicist.
    • Strong families are only peripherally useful if capable of creating the opportunities and support needed to produce a physicist.
    • We are all only a product of our environment. That environment is primarily a result of the opportunities and support given freely by its members. So if your family is not strong, one should look in the mirror first.
    • A plant dies because you did not water it, not because of the room it was placed within.