General: Understanding of CPU/Memory communication

[dsguy]

Hi, I recently finished part 1 of the course and I must say I've been loving it, learned a ton and really looking forward to continuing the journey with part 2.

However, there is still one thing that I cannot wrap my head around. I've been doing quite some research and whenever I thought I understood the concept, then another source popped up with explanations seemingly contradicting what I had thought I understood before.
I am pretty sure I am just not really understanding a core piece of terminology or whatever, so hopefully someone here can clear things up for me.

High level, I am confused about how the CPU and Memory exchange pieces of information of varying width, how the former operates on them and how the latter stores them:

* Computers usually come in architectures of X-bit (usually 32 or 64); at the beginning of this course, when building out all the chips, I was under the impression that a X-bit computer must be build from X-bit parts - is this assumption correct or does it only refer to bus width? E.g. can I have 2X-bit registers in X-bit computer?

* I am using a 64-bit laptop, yet when I hexdump e.g. a file that contains the string "hello", I can see that each letter is stored in one byte - each with its own memory address. This is what I don't understand! Does that mean at some place between the ALU and the Memory there is a "thing" that fans out a single 64-bit instruction into (in this case) 5 instructions? How would that work?

* Memory chips - if I understood correctly - can be much wider than regular computer architectures, like 128, 256 bit wide. If I have a chip with a 256 bit register, is every register simply partitioned into 32 byte parts, each with their own address (through some means)?

* How does a 64-bit CPU operate on multiple bytes retrieved from Memory? Say, the "hello" string stored in Memory is retrieved and displayed on a screen, is this done in a single operation, can the magical "thing" I don't understand take 5 smaller instructions and glue them together to one 64-bit instruction or is it maybe simply done in multiple clock cycles (which seems unlikely for me since it sounds like quite a waste).

* If my assumptions from above are correct, where does Hack fit in here? Within Hack, everything is 16-bit and going back and forth is very logical to me. Is this for simplicity reasons?

At some point during my research I understood that storing information such as a single character in 16, 32 or 64-bit memory locations is of course a total waste of space and storing individual bytes makes much more sense. I just feel like I don't really understand how varying widths of information and physical chips can easily communicate with each other.

Thanks (and sorry for the long post :D)

[WBahn]

You are asking very good questions. To answer one of them -- the Hack is a strictly 16-bit machine and all of the tools that you are asked to develop are also strictly 16-bit. This, very much, is for simplicity.

Having said that, you could write Jack libraries that work with different-sized data. This is done, in fact, in the character output library where information about two characters are stored in the same word. You could write libraries that work with single-byte data, four-byte integers, or 64-bit double-precision floating point numbers. Implementing things at this level is usually referred to as "emulation" -- you are emulating hardware behavior with software. Early PC's had no support for working with floating point representations, so this was done in software.

The degree to which the hardware supports working with different-width data depends very strongly on the processor. In general, the lower the cost of the processor, the more likely it will only support operations on it's native data width. At the other end, processors like the x86-64 have very rich instruction sets that support working on data of several different widths, primarily 8-, 16-, 32-, and 64-bit integers.

You had asked if a processor could have registers that are a different size that it's bus-width. There's nothing that prevents this, but it is uncommon. Instead, smaller-width values use a portion of a full register while wider-width data is split across multiple registers. Having said that, the x86-64 has sixteen floating-point and vector registers that are 256-bits wide.

[WBahn]

I realized I didn't address a couple of things.

You asked about the width of memory being 256 bits but being used on a 64-bit machine. The memory system on a PC is pretty complex, involving several layers of cache. So the processor never actually sees the DRAM chips, instead they transfer data back and forth between the nearest layer of cache (usually L3). Because DRAM is so much slower than the processor, as much data needs to be transferred on each clock cycles as possible. The processor's bus width is not a factor, what matters is that the interface between the DRAM and the L3 cache be matched. The same is true up and down the memory hierarchy.

To expand on my earlier post, the x86-64 is "byte-addressable", meaning that it can access memory starting at any byte boundary. How it does this is by accessing memory in a 64-bit chunk and then performing logic operations, in hardware, to grab the desired byte and put it in the desired register.

Another question you might ask is, if smaller-width data is stored in full-width registers, what happens to the "unused" bits in the register. The answer depends on the instruction. Most of the time, the unused bits are not changed. There are exceptions.

[dsguy]

Edit: I just saw that you have extended your answer while I was writing this up. The fact that different layers of Memory take care of a lot of this "mapping" makes a lot of sense and demystifies it quite a bit. So I guess the first paragraph of this post is not really true since "byte-sized" does indeed exist on the hardware level.

Thanks for the quick reply, WBahn! This already clears up some of my confusion, very helpful.

I think my understanding of what data actually is, is what's confusing me here. Is it correct to assume that when I am looking at "byte-sized" data, this is how programmer-created data is encoded? Simply a set of rules and conventions about how integers, characters etc. are stored?

E.g. (simplified) "a character is encoded using 8 bits; characters are stored in memory segments of exactly 8 bits".
A register within Memory is then loaded with "high" bits in various locations, but this is a different kind of "data" (physical bits in a Memory != 1001 representing int 9)?

If my assumption is correct, does that also mean that, when I am looking at my hexdump output from the original post, characters stored in byte-sized memory locations are simply an encoding as well? Assuming a 64 bit wide Memory chip, is "hello" being stored in a single Memory register and the Memory addresses shown in the hexdump (one per characters) are nothing but an encoding as well?
So somewhere, there is a mapping that says "Memory address 0-4 stores 'hello' and is located in the physical Memory register 0"?

Does the OS take care of this kind of stuff?

[WBahn]

As far as the hardware, be it the processor, the memory, the hard disk, you name it, data is just a bunch of bits stored somewhere. When your program loads a byte of memory into the CPU and does some operation on it, the CPU neither knows nor cares whether that byte represents a character, a number, a part of a multi-byte number, part of a floating point value, a bunch of individual flag bits, or anything else. It's the programs that we write that operate on that data that imposes behavior consistent with a particular interpretation of what those bits mean.

[dsguy]

After some more research, I have come to the conclusion that my confusion derived from a fundamental misunderstanding of the fact that "data bus width is not the same as the smallest addressable memory unit" alongside not knowing of the existence of memory hierarchy and byte-addressability to begin with.

Having read more into the topic and re-reading your answers have really made things clear to me now, so thanks again for the very diligent answers, appreciate it!

Having said that, I figured that this is enough for me to understand of the topic since digging any further into memory hierarchy looks like a massive rabbit hole to go down into, at least at this point.

Thanks!